5 ms·
Does anyone have a resource for getting into app-based scraping, if the API is obfuscated or rate limited?
by Quessked73 5y ago
Does anyone have a resource for getting into app-based scraping, if the API is obfuscated or rate limited?
- elorant 5y agoRun an http debugger, or some proxy, and find the endpoints.
- nlh 5y agoProxyMan on MacOS is quite awesome for this. It requires a bit of setup with certificates, etc., but once it's working you just fire up the target app on your phone and all the sweet sweet API requests appear on your big screen. I've scraped two apps very successfully this way. It's also fascinating to see how developers-who-aren't-me setup their APIs when they assume that nobody's looking.