6 ms·
Who gets to decide what is a good bot?
by pryelluw 2mo ago
Who gets to decide what is a good bot?
- tccole 2mo agoMe… obviously
- nullpoint420 2mo agoCloudflare, apparently.
- mark212 2mo agono, their customers. Why do you assume that people who run websites are clueless and lack agency in this?
- nullpoint420 2mo agoinb4 "we're updating our Terms of Service - all Cloudflare-based scrapers will now automatically be enrolled to scrape your cached websites" and they'll explain how it's great because it's only reading the cached versions, not hitting upstream.
- pythonaut_16 2mo agoPresumably the site owner with Cloudflare providing enforcement. Generally isn't a good bot one that respects robots.txt and is respectful of the site's resources by not being spammy?
- wnevets 2mo agoby checking which follow robots.txt?
- wmf 2mo agoAccording to their plan, the good bots pay for scraping.
- overfeed 2mo ago> Who gets to decide what is a good bot? Paying Cloudflare's tolls => good bot.