Greg Hogg
Region: CA
Thursday 23 October 2025 14:52:04 GMT
38210
1433
24
47
Music
Download
Comments
Zumokute :
🔥🔥🔥
2025-10-23 15:00:53
0
Konstantine :
voice crack of doom and despair 🥀
2025-10-24 04:22:36
139
A :
Do it with a shopping website that has antibot and anti web scraping protections. Web scrapping is easy on websites that don’t care but what about getting info from a website that doesn’t want you to scrape
2025-10-23 16:47:11
115
jaawn :
The hard part of most modern web scraping is not the scraping itself, it is evading anti-scraping protections on the page and making sense of weird page structure in the HTML
2025-10-23 21:27:07
29
llun tes 🇨🇦 :
it's not that simple, especially if the table isn't baked into the GET request of the page. you have to actually simulate the browser in order to actually get the info you want. sites can also detect this that you have to avoid. it's actually not a trivial problem at all.
2025-10-23 18:08:22
15
Jeidoz :
Now try do it with JS content generated sites or virtual DOM sites, where you need await render of data, captcha and API limits. 😅
2025-10-24 13:14:03
10
Jared :
scraping is hard from websites that do not want to be scraped.
2025-10-24 00:01:49
8
derill :
This is obviously bs. Why ? Here is a summary. Static HTML Only — can’t see data loaded dynamically by JavaScript (e.g., most modern sites). No Request Control — you can’t set headers, cookies, or sessions, so many sites block it. Limited Scope — it only reads tags, nothing else on the page. Weak with Messy Tables — merged cells or inconsistent headers easily break parsing. Not Built for Scraping — lacks anti-bot tools, retries, and other resilience features. Basically wikepedia is one of the few website where this will work, most of them will not even accept a request from python code like this.
2025-10-23 18:11:14
4
Sparky :
except when u need to handle pagination 😔
2025-10-23 22:49:51
3
Angel Syangbo :
The only huge misconception here is you thinking this is 99% of web scraping 🥀🥀
2025-10-23 18:55:02
2
Amir H :
anyone that is doing scraping at scale is spending lots of money in proxy headless browsers ..
2025-10-23 23:28:44
1
Eric Orr :
Ehh, it gets complicated fast. Python is too slow for this task at scale, you need rotating IP proxies, you need streaming ingest of page, you need randomization of user agents and cookies, etc etc. And you have to do it all dynamically because nothing worth scraping is served statically.
2025-10-24 09:29:18
1
ImYrMama :
Yeah until the site has anti-scraping mechanisms 😏
2025-10-24 11:06:14
1
user2259260204545 :
now try doing a dynamic webapp...
2025-11-13 16:25:45
1
Cestmoiaussi :
Also chrome has an extension that does it. Can be tricky to learn but works pretty well
2025-10-24 17:22:32
0
TGF1899 :
I wish all I had to web scrape is a table most of the time I need scrape each class and build the table that way and wait hours for the code to run so that it generates the thousands of rows of data
2025-10-24 19:59:22
0
Nas :
not 99% percent of the time..
2025-10-25 00:53:59
0
thetiktokgod :
just pay apify bro
2025-10-26 06:40:03
0
To see more videos from user @greghogg5, please go to the Tikwm
homepage.
