6 ms·

I'm a big fan of modern JavaScript frameworks, but I don't fancy SSR, so have been experimenting with crawling myself for uploading to hosts without having to d

by unlog 2y ago

I'm a big fan of modern JavaScript frameworks, but I don't fancy SSR, so have been experimenting with crawling myself for uploading to hosts without having to do SSR. This is the result

mikeqq2024 2y ago
for a long crawling task, if exited/broken for any reason, does it save and restore at the next run?
- eXpl0it3r 2y ago
  The README says: > Can resume if process exit, save checkpoint every 250 urls
  - mikeqq2024 2y ago
    nice, better make it as a commandline option with default value. 250 is too many for large files and slow connection.