These are AI bots. Computational capacity is not a limiting factor. I'd argue that my desktop consumer PC is less capable of efficiently solving a PoW than a mu

These are AI bots. Computational capacity is not a limiting factor. I'd argue that my desktop consumer PC is less capable of efficiently solving a PoW than a multi-GPU cluster in a data center. Even if, as you say, crawlers will hit the PoW thousands of times more, the only way to make it a barrier is if the cost is higher than the profit to be gained. Otherwise it's merely an expense to be passed on to the customer.