7 ms·
Just because you can do it without much effort doesn't mean less experienced people can. Crawling can be a barrier to some people.
by frederi 13y ago
Just because you can do it without much effort doesn't mean less experienced people can. Crawling can be a barrier to some people.
- boyter 13y agoTo be honest a simple crawler is a very simple thing to write. If someone had issues getting that going I think they are going to have issues with the data volume anyway. LisaG answered why the 3 million data set though and I agree with the reasoning.