6 ms·
Cloudflare specifically has a block for LLM and AI training bots now. Not sure of the effectiveness but it's there.
by Grimburger 1mo ago
Cloudflare specifically has a block for LLM and AI training bots now.
Not sure of the effectiveness but it's there.
- matherial 1mo agoMinimal. I'm behind Cloudflare and 90% of the traffic is still scrapers. I don't think they're serious about the long tail. I think the main thing Cloudflare is trying to do is block direct traffic from frontier labs and then start charging them for access. They might end up shooting themselves in the foot, as this simply empowers sketchy residential-proxy outfits to undercut Cloudflare and sell the data to labs for less.
- inigyou 1mo agoI think the other thing they're trying to do is get most of the internet to send them all of their cleartext traffic. Expect in 2040 the PRISM2 docs will get leaked by some Eduardo Rainedon and we'll find out Cloudflare was the NSA all along.
- bakugo 1mo agoIt still only blocks "well-behaved" bots that have proper User-Agents and respect robots.txt, so it's largely pointless. The problematic bots are all disguising themselves as Chrome and sending requests from millions of residential proxy IPs, and the only real solution to those is some sort of captcha or PoW page on first visit.
- satvikpendem 1mo agoWhile they themselves announced an AI bot, the irony is palpable. In reality they just want to control who gets access to what, themselves excepted.