buffer_overflow

6 ms·

1. 1.▲
   Show HN: MrScraper AI – Dead simple web scraper (powered by AI)(mrscraper.com)

   38 points by buffer_overflow 3y ago|27 comments
2. 2.▲
   by buffer_overflow 4y ago

   Interesting project, Tom! I should plan more before coding. Is Code Planner free or paid?
3. 3.▲
   by buffer_overflow 4y ago

   Already tried that. Not worth it with the current token limitations. But I'm currently tinkering with other applications of AI and MrScraper. Will ship something when I think it's reliable enough.
4. 4.▲
   by buffer_overflow 4y ago

   I'm experimenting with something like you described, but there are lots of edge cases and needs a bit of polishing. But for now, I made a Chrome extension that helps tinkering with CSS selectors.
5. 5.▲
   by buffer_overflow 4y ago

   The onboarding process is not a fillable form, it's just a simplification of the scraper builder to show how it works. You just have to click next. Other users have also reported that this is a bit confusing, so I'm going to start
6. 6.▲
   by buffer_overflow 4y ago

   If the target page or DOM changes, it's not considered a failed scraping, just the data you wanted will return empty. But you gave me the idea to send an alert or email notification if a scraper stops returning data or content changes.
7. 7.▲
   by buffer_overflow 4y ago

   Thanks for your precious feedback! I've noted everything down and I'll improve the onboarding experience, fix and make variable names more understandable and improve the edit button. A scraper is not editable once is queued to run
8. 8.▲
   by buffer_overflow 4y ago

   I could depending on the proxy I got from my provider. I'm currently working on adding the ability to select a higher quality proxy for difficult to scrape websites and to add captcha solvers as well. But if I have to be honest, I can
9. 9.▲
   by buffer_overflow 4y ago

   Honestly I haven't tried Octoparse, but doing a quick check, I can see it is way more expensive.
10. 10.▲
    by buffer_overflow 4y ago

    Sorry for not being more clear. The onboarding process is a simplified version of the actual scraper builder. The fields are not editable, it's just to get used to the scraping flow. I've noted down your suggestion and I'll m
11. 11.▲
    by buffer_overflow 4y ago

    Thanks for this amazing feedback. I can get some action items from this. I've noted down your comment and I'll be improving things for next week!
12. 12.▲
    by buffer_overflow 4y ago

    Thank you so much. Appreciate it!
13. 13.▲
    by buffer_overflow 4y ago

    Hi there, I'm currently working on standard pagination (click next page button) and click button + infinite scroll. What you comment is not currently possible with a single scraper, you would need to send one to collect links and then
14. 14.▲
    by buffer_overflow 4y ago

    Thanks for reporting. I'll review this!
15. 15.▲
    by buffer_overflow 4y ago

    Thanks for sharing your point of view! I will rewrite the copy to make better statements. Thank you so much
16. 16.▲
    by buffer_overflow 4y ago

    Noted! I'll improve this
17. 17.▲
    by buffer_overflow 4y ago

    Thanks, I will do one!
18. 18.▲
    by buffer_overflow 4y ago

    Honestly, I've been working on it for two months and didn't reach this part of the roadmap yet. I was planning to use a 2captcha integration for the first approach.
19. 19.▲
    by buffer_overflow 4y ago

    Thanks buddy!
20. 20.▲
    Show HN: MrScraper – A visual web-scraping tool(mrscraper.com)

    215 points by buffer_overflow 4y ago|82 comments