Quatschmann

7 ms·

1. 1.▲
   by Quatschmann 10y ago

   I guess Netherlands.
2. 2.▲
   by Quatschmann 10y ago

   Yep, that's why I've build my own, the existing ones don't give out a list of the links or are super slow. A co-worker made the first one in Python but it was so slow that it took hours (6+ sometimes) to finish a site and I
3. 3.▲
   by Quatschmann 10y ago

   I wrote a concurrent, super fast webcrawler for my job with Go (~300 LOC) to get data out of customer sites fast even when they have 1.5 million pages or more. You can basically filter everything to get a .csv file in the end with the links