15 ms·

But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine? They can just use PhantomJS (http:

But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine? They can just use PhantomJS (http://phantomjs.org/ http://phantomjs.org/), which is free and open source.

voltagex_ 12y ago
They could, but I wonder what it'd take to scale it to crawling that number of pages. I think only Bing would have the cash and resources to build that.