5 ms·Robots.txt? Just send a fake user agent.by CommanderData 2y agoRobots.txt? Just send a fake user agent.