11 ms·
you have to feed it multiple arguments with rate limiting and long wait times. i am not sure if there have been recent updates other than the js interpreter but
by fibers 1y ago
you have to feed it multiple arguments with rate limiting and long wait times. i am not sure if there have been recent updates other than the js interpreter but ive had to spin up a docker instance of a browser to feed it session cookies as well.
- m_ke 1y agoYeah we had to roll through a bunch of proxy servers on top of all the other tricks you mentioned to reliably download at a decent pace
- trenchpilgrim 1y agoWhat are your thoughts on the load scrapers are putting on website operators?
- immibis 1y agoWhat are your thoughts on the load website operators are putting on themselves to block scrapers?