4 ms·
Claude seems to follow robots.txt by default. Actually at my organization our theory is that this is why no one is finding our public results any more.
by dguest 5d ago
Claude seems to follow robots.txt by default. Actually at my organization our theory is that this is why no one is finding our public results any more.