A better solution might be to throttle multiple requests to the same domain. Also, they could prevent it from fetching the same content multiple times and use a

A better solution might be to throttle multiple requests to the same domain. Also, they could prevent it from fetching the same content multiple times and use a cache instead based on ETAGs and file information.