5 ms·

$12k/month SaaS scraping social networks’ data

37 points by richclominson 7y ago

PatrolX 7y ago
Not the first to do it and the last one I know of that did got a severe beating in court and they lost a lot of money.
- teddyuk 7y ago
  How easy is it for a us company to do anything about a Ukraine’ian company?
  - b212 7y ago
    Extremely hard without shady techniques and/or bribes.
  - deleted 7y ago
    [deleted]
kyleblarson 7y ago
Cue the cease and desist letter in 3,2,1...
kryogen1c 7y ago
@dang are we ok with this? This account is basically an automated submission script for this website. Submission history is nothing but. I dont mind people promoting themselves or their work, but i do not come to HN for ad aggregation or an RSS feed.
- minimaxir 7y ago
  See also starterstory in the same industry: https://news.ycombinator.com/from?site=starterstory.com https://news.ycombinator.com/from?site=starterstory.com
- dang 7y ago
  You should email hn@ycombinator.com to ask questions because we don't see most comments like this. Someone else emailed me about it. Accounts that use HN just to promote their own stuff eventually get penalized or lose submission privileges on HN. People are welcome to submit their own things as part of a variety of stories, including unrelated ones that they ran across personally and gratified their intellectual curiosity. That's the best way to participate in the community.
- richclominson 7y ago
  Hey! It's the submitter here. I actually don't know how to code so I promise I have no script. I just go and share the stuff I publish here because I think that some people may find it interesting and useful as it happened with this interview. From now on, I will be more careful with the amount and frequency of posts I share. Sorry for the inconvenience.
  - kryogen1c 7y ago
    Id be more inclined to believe you if this wasn't the first time you had commented in two years.
nif2ee 7y ago
How is this even legal?
- deusofnull 7y ago
  "in Jersey[1] everythings legal as long as you dont get caught[2]" - bob dylan in 'tweeter and the monkey man' by the traveling willburys. [1] replace jersey with ukraine [2] replace caught with extradited
- sdinsn 7y ago
  Scraping public information is legal.
minimaxir 7y ago
A common trend I've seen recently on HN/Reddit is that due to the ruling of the HiQ vs. LinkedIn court case (https://news.ycombinator.com/item?id=21241395 https://news.ycombinator.com/item?id=21241395), data scraping cannot be against the ToS of a given service. That is not universally correct. And even if it is, selling the data is an easy way to an expensive lawsuit.
deusofnull 7y ago
funny, i build a basic little script that did this back in like 2015 so i could give my GF at the time a copy of all her instagram, which she was trying to do via scrolling + screenshots. I was thinking about expanding it but didnt. Thought it would be cool if you could order a literal physical book with all your IG photos in it. Kinda a gag gift but probably worth the cost of a modest gift (im thinkin like 20-100 bucks). Never got to the book part. Def would have run into legal troubles, but I guess if youre abroad (Ukraine?) then the US judicial system is not too much a concern.
dewey 7y ago
Don’t these services have a “takeout” option like Twitter, Google? Where you can download everything for free?
- minimaxir 7y ago
  Facebook does, Instagram does not.
  - wouterinho 7y ago
    This is not true, Instagram also offers this: https://www.instagram.com/download/request/ https://www.instagram.com/download/request/
    - minimaxir 7y ago
      huh, TIL. I apologize. "Privacy and Security" is an odd place for that menu.
  - dewey 7y ago
    As it's an EU law a bunch of companies have this automated. In the EU you can request the data that a company has on you from any company via a letter / form. https://ec.europa.eu/info/law/law-topic/data-protection/reform/rights-citizens/my-rights/how-can-i-access-my-personal-data-held-company-organisation_en https://ec.europa.eu/info/law/law-topic/data-protection/refo...
atum47 7y ago
https://github.com/victorqribeiro/scrap/blob/master/scrap.js https://github.com/victorqribeiro/scrap/blob/master/scrap.js I wrote a Script that does something similar. In this case it is getting peoples name and image address (from facebook). I don't remember why I did this, but I never thought of making money out of it

37 points by richclominson 7y ago

PatrolX 7y ago

Not the first to do it and the last one I know of that did got a severe beating in court and they lost a lot of money.

teddyuk 7y ago
How easy is it for a us company to do anything about a Ukraine’ian company?
- b212 7y ago
  Extremely hard without shady techniques and/or bribes.
- deleted 7y ago
  [deleted]

kyleblarson 7y ago

Cue the cease and desist letter in 3,2,1...

kryogen1c 7y ago

@dang are we ok with this? This account is basically an automated submission script for this website. Submission history is nothing but. I dont mind people promoting themselves or their work, but i do not come to HN for ad aggregation or an RSS feed.

minimaxir 7y ago
See also starterstory in the same industry: https://news.ycombinator.com/from?site=starterstory.com https://news.ycombinator.com/from?site=starterstory.com
dang 7y ago
You should email hn@ycombinator.com to ask questions because we don't see most comments like this. Someone else emailed me about it. Accounts that use HN just to promote their own stuff eventually get penalized or lose submission privileges on HN. People are welcome to submit their own things as part of a variety of stories, including unrelated ones that they ran across personally and gratified their intellectual curiosity. That's the best way to participate in the community.
richclominson 7y ago
Hey! It's the submitter here. I actually don't know how to code so I promise I have no script. I just go and share the stuff I publish here because I think that some people may find it interesting and useful as it happened with this interview. From now on, I will be more careful with the amount and frequency of posts I share. Sorry for the inconvenience.
- kryogen1c 7y ago
  Id be more inclined to believe you if this wasn't the first time you had commented in two years.

nif2ee 7y ago

How is this even legal?

deusofnull 7y ago
"in Jersey[1] everythings legal as long as you dont get caught[2]" - bob dylan in 'tweeter and the monkey man' by the traveling willburys. [1] replace jersey with ukraine [2] replace caught with extradited
sdinsn 7y ago
Scraping public information is legal.

minimaxir 7y ago

A common trend I've seen recently on HN/Reddit is that due to the ruling of the HiQ vs. LinkedIn court case (https://news.ycombinator.com/item?id=21241395 https://news.ycombinator.com/item?id=21241395), data scraping cannot be against the ToS of a given service. That is not universally correct. And even if it is, selling the data is an easy way to an expensive lawsuit.

deusofnull 7y ago

funny, i build a basic little script that did this back in like 2015 so i could give my GF at the time a copy of all her instagram, which she was trying to do via scrolling + screenshots. I was thinking about expanding it but didnt. Thought it would be cool if you could order a literal physical book with all your IG photos in it. Kinda a gag gift but probably worth the cost of a modest gift (im thinkin like 20-100 bucks). Never got to the book part. Def would have run into legal troubles, but I guess if youre abroad (Ukraine?) then the US judicial system is not too much a concern.

dewey 7y ago

Don’t these services have a “takeout” option like Twitter, Google? Where you can download everything for free?

minimaxir 7y ago
Facebook does, Instagram does not.
- wouterinho 7y ago
  This is not true, Instagram also offers this: https://www.instagram.com/download/request/ https://www.instagram.com/download/request/
  - minimaxir 7y ago
    huh, TIL. I apologize. "Privacy and Security" is an odd place for that menu.
- dewey 7y ago
  As it's an EU law a bunch of companies have this automated. In the EU you can request the data that a company has on you from any company via a letter / form. https://ec.europa.eu/info/law/law-topic/data-protection/reform/rights-citizens/my-rights/how-can-i-access-my-personal-data-held-company-organisation_en https://ec.europa.eu/info/law/law-topic/data-protection/refo...

atum47 7y ago

https://github.com/victorqribeiro/scrap/blob/master/scrap.js https://github.com/victorqribeiro/scrap/blob/master/scrap.js I wrote a Script that does something similar. In this case it is getting peoples name and image address (from facebook). I don't remember why I did this, but I never thought of making money out of it