It could make sense if it's checking for robots.txt or so, though I'm not sure if it does that each time search results and I would have thought that that would

It could make sense if it's checking for robots.txt or so, though I'm not sure if it does that each time search results and I would have thought that that would be conducted from the server side rather than client side. Are you sure you don't have some browser extension that does some checks/inspection/etc of links?