20 ms·
Scraping webpages is extremely useful and this would seem to combat this. It's also extremely useful by... oh yes... Google. And I'm sure they would find a way
by afs35mm 3y ago
Scraping webpages is extremely useful and this would seem to combat this. It's also extremely useful by... oh yes... Google. And I'm sure they would find a way to whitelist their scrapers to index pages, but archive.org? Oh you're SOL.