6 ms·
Most people are probably using scrapers and crawlers to collect data on sites that either don't have an open API or the API is difficult to use. Examples: Anc
by mjhea0 7y ago
Most people are probably using scrapers and crawlers to collect data on sites that either don't have an open API or the API is difficult to use.
Examples:
Ancestry.com scraper - https://github.com/mjhea0/ancestry-scraper https://github.com/mjhea0/ancestry-scraper
Indeed job scraper - https://github.com/mjhea0/indeed-scraper https://github.com/mjhea0/indeed-scraper
Craigslist housing scraper - https://github.com/mjhea0/craigslist-housing-scraper https://github.com/mjhea0/craigslist-housing-scraper
https://github.com/ThaWeatherman/scrapers https://github.com/ThaWeatherman/scrapers
What can you then do with the data?
Monitor competition or your own brand
Sentiment Analysis
Gather sales leads
Machine learning
Generate content (blog posts, building a custom job board)
Find cheap flights