7 ms·
Actually in the password reset cases you should use noindex. In the case of the expensive page that you don't want bots crawling, the only sure fire way is to
by nottrobin 4y ago
Actually in the password reset cases you should use noindex.
In the case of the expensive page that you don't want bots crawling, the only sure fire way is to block bots based on the user agent string.
Google both follows and indexes nofollow links.