6 ms·
They could in theory combat it by comparing results with a second crawler that uses a different User Agent.
by dmitrysergeyev 3y ago
They could in theory combat it by comparing results with a second crawler that uses a different User Agent.
- mdaniel 3y agoIf they were going to the amount of energy to do a 2nd crawl using a different user agent, then why bother advertising the user agent at all and just feed it the Chrome one like every other home-grown spider does
- deleted 3y ago[deleted]