rustdeveloper

6 ms·

1.
by rustdeveloper 2y ago
Happy to suggest another web scraping API alternative I rely on: https://scrapingfish.com
2.
by rustdeveloper 2y ago
Interesting data for sugar in products scraped from Walmart: https://scrapingfish.com/blog/scraping-walmart
3.
by rustdeveloper 2y ago
This is correct, my friends from Scraping Fish are hosting https://compareproxy.com to help people find proxy for web scraping. I'm happy to "push" for Scraping Fish as I'm also a satisfied user who received
4.
by rustdeveloper 2y ago
And many more: https://compareproxy.com/
5.
by rustdeveloper 2y ago
For web scraping at scale you want to get lost in the crowd. This usually means being (or pretending to be) chromium on windows. Unusual browsers are suspicious, detected or have very distinct fingerprint.
6.
by rustdeveloper 2y ago
This is a terrible news :( I know it was an option for web scraping and I used in once. I’m curious what is the real reason they took it down.
7.
by rustdeveloper 2y ago
Surprisingly, according to you tool, HN is neutral on “web scraping”. I noticed others also reported bias for neutral on other keywords.
8.
by rustdeveloper 2y ago
There are also SaaS products with usage based pricing. It depends on what SaaS or software or product it is. Different pricing model works for different things.
9.
by rustdeveloper 2y ago
Actually, they do allow this. I store my photos and iPhone backup on synology NAS. You just have to devote some time to set it up yourself.
10.
by rustdeveloper 3y ago
I'm using Scraping Fish because of their pay-as-you-go style pricing as opposed to subscription with monthly scraping volume commitment. And they don't charge extra credits for JS rendering or residential proxies because the cost
11.
Bulk Google Index Checker(serpnames.com)
1 points by rustdeveloper 3y ago|0 comments
12.
by rustdeveloper 3y ago
This reminds me of https://scrapingfish.com/blog/are-most-rust-jobs-in-crypto :)
13.
by rustdeveloper 3y ago
Do we know how LLMs available in OpenLLM and other open source LLMs compare to different versions of GPT models? I know there’s a leaderboard on huggingface: https://huggingface.co/spaces/HuggingFaceH4/open_llm_lea
14.
by rustdeveloper 3y ago
Does it give access to the internet? Could it be used with this setup: https://scrapingfish.com/blog/byo-mobile-proxy-for-web-scrap... ?
15.
by rustdeveloper 3y ago
For $20 you can get over 10k requests from scraping fish: https://scrapingfish.com/buy
16.
by rustdeveloper 3y ago
I don’t see how any LLM would help me with a high quality proxy, which is what I actually need in web scraping and I’m using https://scrapingfish.com/ for this.
17.
by rustdeveloper 4y ago
There are tutorials for building you own mobile proxy pool so it’s very accessible these days: https://scrapingfish.com/blog/byo-mobile-proxy-for-web-scrap...
18.
by rustdeveloper 4y ago
I'll be working on my own mobile proxy build: https://scrapingfish.com/blog/byo-mobile-proxy-for-web-scrap...
19.
by rustdeveloper 4y ago
I initially built a system for web scraping but was constantly running into issues of getting blocked, even when using good quality residential proxies. I had to constantly investigate why I'm getting blocked and update tools. Sometime
20.
by rustdeveloper 4y ago
This looks really cool! There was a tutorial posted on HN about building mobile proxy pool with RPI that had obvious limitations: https://scrapingfish.com/blog/byo-mobile-proxy-for-web-scrap... It seems this could be a
21.
by rustdeveloper 4y ago
For web scraping, I recommend using a web scraping API, e.g. https://scrapingfish.com . This solves all potential problems with getting blocked and can make data extraction easier as well. For the app, I've recently started
22.
by rustdeveloper 4y ago
I use Scraping Fish API: https://scrapingfish.com/
23.
by rustdeveloper 4y ago
“This industry makes scraping available to individuals and companies that otherwise would not have the capabilities.” - seems like web scraping companies are doing a good job :)
24.
by rustdeveloper 4y ago
Looks interesting. Will try the script for web scraping.
25.
by rustdeveloper 4y ago
I think that if you don’t want to invest a lot of time into learning web scraping and money to get a pool of residential, or even better mobile, proxies it’s easy to quickly get good results with web scraping API like https://scr
26.
by rustdeveloper 4y ago
I don’t see this happening. If you know rust you can use it elsewhere.
27.
by rustdeveloper 4y ago
I didn’t find so many crypto job offers for python. But your right, I’m curious how it looks for go or cpp.
28.
by rustdeveloper 4y ago
I was looking for a rust job recently and got the same impression that most of them are in crypto :|