7 ms·
There are a bunch of comments about rolling your own scraper instead of relying upon a possibly unreliable SaaS app. That makes me think -- would it be viable
by RaphiePS 12y ago
There are a bunch of comments about rolling your own scraper instead of relying upon a possibly unreliable SaaS app.
That makes me think -- would it be viable to run a service that, instead of running the scraping on their own servers, simply gave you a custom binary to run?
Assuming that you trusted the executable, you would never have to worry about the company failing. It'd just be a one-time fee, and yours to use in perpetuity. Presumably updates would be free.
- caio1982 12y agoThat's a really neat idea I'd pay for. Not sure about the sustainability of the model though.
- hyuuu 12y agoif you use scrapy (which is an awesome python scraping framework) you can plug a different third party solutions such as: http://crawlera.com/ http://crawlera.com/ Not really server level hosting, but you get the benefits of their network.