@tech.bible: Turns out you can actually put a paywall in front of them. Here are 4 tools for paid crawling to use: AWS WAF : match AI bot traffic, price specific routes, return 402 Payment Required, verify payment and gate access at the edge. Skyfire : let AI agents pay for APIs, tools and data automatically. TollBit : set rates for AI crawlers and charge them for access to your content. Cloudflare Pay Per Crawl: set a price per successful crawl instead of just allowing or blocking the bot. So instead of just allow or block, there’s a third option now: pay me. Still wondering why #vercel doesn’t have a native “charge this bot” toggle in the dashboard yet. Guys… pls 😭 I’m Ghita, I build in AI, go way too deep into infrastructure rabbit holes and share what I learn. And if you want to understand what software + AI your company is actually using, what it costs and where you can save, check out Eli.work
Ghita 🤖
Region: GB
Monday 12 October 2026 15:09:53 GMT
Music
Download
Comments
Javito.sibauste :
ok but what are the benefits of having a website crawl so many times?
2026-10-12 19:39:46
27
velostevie :
Yup - I have Plausible.io telling me I have maybe 16 visitors in a day, but Cloudflare says it’s 3000 users making 17,000 requests.
2026-10-12 16:07:09
399
Nikolaj :
I will consider a tarpit to trap the AI's and just spend their cpu
2026-10-12 20:08:21
178
Alex Santos :
Will pay per crawl limit website ranking for example?
2026-10-12 20:08:30
40
Wandering_Mike_REAL :
PAY PER CRAWL LETS GOOOOO
2026-10-13 02:42:40
97
archie :
Is that what that cloudflare thing is for
2026-10-12 17:15:09
42
Ben Blakley :
what payment method will the bots pay with?
2026-10-13 07:14:11
12
Gareth :
I’ll try it 😅 never heard of this before though haha
2026-10-12 15:54:37
6
peanutsandraisinswtchoco :
Can someone explain this to me in layman terms please?
2026-10-13 09:55:23
1
maddy13377 :
How do you find out or
2026-10-12 15:54:29
1
Andy :
You fix that cloudflare settings, it will dissapear
2026-10-13 09:27:45
5
Jordan :
no wonder I didn't notice it's vercel. never seen it In light mode
2026-10-13 00:20:36
1
gabriel🔥 :
there's ZERO tools that will not also annoy real customers tho.
2026-10-13 15:25:55
1
imtiredgoingtobedsoon :
those are bots if it's meta
2026-10-13 15:27:37
1
Meeeeepo :
welcome to the club buddy 😏
2026-10-15 06:26:50
0
Orgest :
Good advice
2026-10-12 16:19:33
1
yoshi_42 :
I've put anubis in my gitea instance, crawlers were hammering my instances
2026-10-13 01:49:22
1
maddy13377 :
Thanks
2026-10-12 15:54:41
0
almafeta :
that's nothing tho?
2026-10-14 00:41:59
0
Schafer :
AI dropshipping is teaching me that patience and proper learning are important when starting an online business.
2026-10-14 00:04:46
0
💚💡Reena💡💚 :
a big space for x402
2026-10-14 18:52:50
0
atlanticperspectives :
Yeah this will take a dump on your AEO though. If you care about that
2026-10-15 11:29:19
0
TheTravelsBlogg🌍TravelBlogger :
I just started building a Wordpress website, what does this mean for it? 😅😂
2026-10-14 07:20:39
0
Sam :
If this is a problem for you then your site isn’t built or hosted correctly
2026-10-14 17:56:59
0
headachesxoxo :
Interesting approach tho! I like the way you think! :)
2026-10-12 21:13:55
0
To see more videos from user @tech.bible, please go to the Tikwm
homepage.