@wesbos: Common way to scrape data from websites that don't provide an API #javascript

Wes Bos Fullstack JS+CSS Tips
Wes Bos Fullstack JS+CSS Tips
Open In TikTok:
Region: CA
Thursday 19 December 2024 19:36:12 GMT
34289
1960
54
102

Music

Download

Comments

webdevsnest
WebDevsNest :
That’s genius. Does this bypass robots.txt and scraping laws?
2024-12-20 22:27:18
0
carl_0zen
carl_0zen :
You can intercept xhr requests as well
2024-12-19 20:45:47
1
samael_lu
Lucifer :
No API just find where they keep the db credentials 😂😂😂
2024-12-19 21:20:50
0
nobodywantedthisusername
ExHummus :
Doesn’t that require js rendering?
2024-12-20 00:19:15
1
neak_programming_geek
Domo :
That’s a clever approach. Normally I code against the xpath. Will try using your way next time. Thanks for the idea
2024-12-19 19:50:43
4
crystalinear
Crystal :
how do you deal with CloudFlare/Captcha? Pretty common to run into that when attempting to scrape something
2024-12-20 10:31:41
15
nice6029
jack :
would this be more or less prone to breaking when a website decides to change it's design? like you mentioned, I would normally just load the whole html, but layout changes break that
2024-12-20 17:35:16
0
anonymously5152
Anonymously :
Any advise for load testing a ui website. I need to load the page wait for content to load... Click ui element that zooms the image.... It doesn't seem to send a request when doing so.
2024-12-22 10:09:54
0
To see more videos from user @wesbos, please go to the Tikwm homepage.

Other Videos


About