A year or two ago I personally encountered scraping bots that were scraping every possible resultant page from a given starting point. So if it scraped a search

A year or two ago I personally encountered scraping bots that were scraping every possible resultant page from a given starting point. So if it scraped a search results page it would also scrape every single distinct combination of facets on that search (including nonsensical combinations e.g. products that match the filter "products where weight<2lbs AND weight>2lbs") We ended up having to block entire ASNs and several subnets (lots from Facebook IPs, interestingly)