30 ms·
I have a personal site on a shared host that was recently suspended for excessive CPU use caused by incessant AI bot crawling. I don't have a problem with crawl
by extra88 2mo ago
I have a personal site on a shared host that was recently suspended for excessive CPU use caused by incessant AI bot crawling. I don't have a problem with crawling per se, there's just so many and they're so bad at it.