4 ms·
It is cloudflare that blocked us, we scrape using a regular chrome instance (with the latest user-agent), but cloudflare are sometimes very aggressive in their
by lpellis 2y ago
It is cloudflare that blocked us, we scrape using a regular chrome instance (with the latest user-agent), but cloudflare are sometimes very aggressive in their blocking.
I'm busy adding a proxy option to handle these cases automatically, and some way to whitelist our scraper.