9 ms·

U.S. tech companies are building a Chinese-style social credit system

127 points by neuro 7y ago

cameronbrown 7y ago
In China, the government has the ability to make me disappear whereas a silicon valley corp. can't. That's the major difference here.
- Finnucane 7y ago
  True, they can just ruin your life, but at least they can't just kill you. That makes it totally okay.
  - cameronbrown 7y ago
    That's at least something citizens can act upon. In China they'll just bury you.
    - anamexis 7y ago
      Just like citizens can act up on, say, getting blackballed by Google and having their email gone forever?
      - greenyoda 7y ago
        It's easy for citizens to fight back against that - by not using Google as their email provider (and educating others about the risks). There are many free and paid alternatives to Gmail.
      - vorpalhex 7y ago
        Are you comparing losing your email versus being kidnapped and held in jail without access to a lawyer or courts?
        anamexis 7y ago
        No, I'm saying this is a thing that deeply impacts people's lives which they cannot, in fact, "act upon" when they are wronged.
        cameronbrown 7y ago
        Take regular backups has always been good advice, that doesn't go away just because a company handles your email for you. Always take archives from cloud services just like you backup HDDs, use a custom domain to forward inbound email traffic and you'll be 100% protected. You chose the worst example of Google being able to ruin somebody's life when if they really wanted to be evil, they'd have a monumental amount of data for blackmail, the ability to silence you online, etc..
- dlphn___xyz 7y ago
  just wait til the red flag law gets passed
  - neonIcon 7y ago
    Careful there dlphn__xyz... I'd hate to have to report you /s
- throwawaymanbot 7y ago
  Still Not a valid reason to allow this abomination to happen here. You dont think that some Govt Dept wont strong arm this technology to do things to you and everyone who supports you?
Vaslo 7y ago
"The most disturbing attribute of a social credit system is not that it’s invasive, but that it’s extralegal. Crimes are punished outside the legal system, which means no presumption of innocence, no legal representation, no judge, no jury, and often no appeal. In other words, it’s an alternative legal system where the accused have fewer rights." Also, what if you disagree with me politically about illegal immigration - does that give you a "right" to ban me from your establishment? Imagine it on the other side as well for people who are against abortion.
- astura 7y ago
  Id imagine I wouldn't be welcome in a church or mosque because of my views on abortion.
  - hprotagonist 7y ago
    you imagine incorrectly. it’s trivially easy to find faith communities with many views on that issue.
    - lliamander 7y ago
      Respectfully, there are faith communities that hold any number of views without regard to their consistency with founding documents or traditions of teachings. I heartily recommend that for anyone who is genuinely seeking spiritual guidance to avoid religious establishments that do not challenge present cultural norms and mores. However, it is very unlikely that a church is going to keep someone out, especially a non-believer, because of their views on abortion. Church attendance is generally open to everyone. It is participation in the sacraments (baptism, communion, etc.) and leadership that often requires doctrinal commitments.
      - hprotagonist 7y ago
        respectfully, that sounds like a fairly mealy-mouthed implicit endorsement of a very specific 19th century form of textual literalism to me. Speak plainly when you state a position, and I'll be happy to return the favor. If i'm reading you properly, you ought to know beforehand that we have probably very different hermeneutics. I would be happy to start on this issue by pointing to any number of long-running (like, millenia-long) ideas on this and other complex moral issues that are littered throughout the many texts of abrahamic faiths. Claiming that there is exactly one intepretation that's valid (which you implicitly do) is a crabbed and limiting way of reading and living pretty much all of the time. On this specific front, my church only recognizes two sacraments, and both are open to all. With regards to challenging present norms and mores, I am happy to agree, albeit in a way that you almost certainly don't mean. For instance: the more of the body of christ that locks themselves to concentration camp gates, the better!
mytailorisrich 7y ago
Insurances assessing risk based on publicly available information (social media posts) is in no way comparable to a social credit system. PatronScan looks potentially more dangerous, but the law in the US and UK (for example and afaik) is that you are free to refuse service to anyone you please as long as it is not illegal discrimination (e.g. in the UK, race or sexual orientation). Once again what this highlights is the power gained by these online platforms. On the one hand as private companies they have no obligation of universal service, on the other hand some of them have so much power that being excluded has a real impact on people. This reinforces my opinion that either these tech giants will effectively rule, or they will have to be controlled in a way similar to what China does in order to keep decisions on censorship, exclusion, and provision of service within public hands.
- mieseratte 7y ago
  > but the law in the US and UK (for example) is that you are free to refuse service to anyone you please as long as it is not illegal discrimination (e.g. in the UK, race or sexual orientation). There is more to discrimination law, including "gotchas" like adverse impact. > or they will have to be controlled in a way similar to what China does in order to keep decisions on censorship, exclusion, and provision of service within public hands. The Western nations seem more concerned with who and what is banned from a platform, the Chinese the opposite, banning speech the Party doesn't like and, perhaps more in line with the West, leveraging these companies as intelligence assets. In this way, the Silicon Valley companies themselves are more like the Chinese whereas the current administration less so.
  - mytailorisrich 7y ago
    The point is that access to many platforms has become important enough that private companies should not be the ones calling the shots. This includes access to service and censorship. I fear that we have opened the pandora box and now the alternatives are those I mentioned. > There is more to discrimination law, including "gotchas" like adverse impact. Not in the UK, unless mandated by law in specific industries.
throwaway_law 7y ago
Isn't this already done to a great extent through black balling? SV recruiters blackball candidates/potential employees (often for no reason than petty vindictiveness); SV incubators/VCs use blackball lists; even insurance has internal lists in order to assess and deny claims based on nothing other than the "social credit" of the claimant rather than the merits of the claim itself. They even go a step further and insurance will rate/rank a claimant's attorney, so you may have a good claim that gets denied because you have a low ranking attorney, or you may have a weak claim they approve because your attorney is highly ranked (probably has a number of jury verdicts in similar cases).
- mountainofdeath 7y ago
  I have never heard of a blackball list at large SV companies unless it was something egregious that person did. Things like starting a fight with security, coming to an interview drunk, etc. will get you on it.
  - thecleaner 7y ago
    >coming to an interview drunk, etc. will get you on it. Really what about the SV party culture ? And if the companies are that professional and only care about your work why does the age bias exist ?
    - TulliusCicero 7y ago
      > Really what about the SV party culture ? What party culture? I worked at Google in Mountain View and never saw any of this, nor "brogrammers", nor endless mandated/encouraged overtime, or most of the other stereotypes that seem to perpetuate themselves.
      - thecleaner 7y ago
        >I worked at Google in Mountain View and never saw any of this, nor "brogrammers"... How long did you work there >5 or >10 years or sth like that or just a stint ?
lidHanteyk 7y ago
Didn't this happen before with some group called "Klout" or some such? I don't remember what happened to them.
- the_watcher 7y ago
  Purchased by Lithium Technologies in 2014, shuttered in 2018. [0] [0] https://en.wikipedia.org/wiki/Klout https://en.wikipedia.org/wiki/Klout
- freehunter 7y ago
  They were bought for their machine learning algorithms and the purchaser ditched the rating service because they recognized it was a bad idea.
andrerm 7y ago
Why do you think Google is indexing all your online purchases? Look, online tracking inevitably leads to behavioral profiling. Put two behavioral profiles side by side and you have scoring, ranking. Add some weight on behaviors, age, income etc. and you have a social credit system. Edit: the question is not if big tech already have social scores in us or not. The questions are when and how they will use it
- word-reader 7y ago
  Exactly; let's say you have a social media record or even a browsing history that indicates labor activism (or drug use, or health issues, or any other kind of activism). That data is saved somewhere, associated with you directly or just assembled into a behavioral profile for ad targeting purposes -- at least for now and as far as we can tell. And like racial bias, this data can be dumped into a neural net with a score at the end to launder what exactly is going on.
- goatinaboat 7y ago
  Why do you think Google is indexing all your online purchases? See https://www.indiatimes.com/technology/news/google-scans-your-gmail-for-invoices-receipts-to-track-what-you-by-online-367598.html https://www.indiatimes.com/technology/news/google-scans-your...
mountainofdeath 7y ago
Fundamentally, for a service to not be discriminatory, it must be regulated like a utility (e.g. telephone, electric, etc). Using something that has the same functionality as a utility but is not regulated as such raises some questions. Also, profiling social media is nothing new. It's not uncommon for insurance companies to hire private investigators to look into suspicious cases. One case I heard was a man going on disability citing being homebound but the private investigator found evidence completely to the contrary. I fail to see how this is any different.
caymanjim 7y ago
Your FICO score and detailed credit history already serve this purpose to a large degree. Everyone knows that affects whether or not you can get a loan (and thus purchase a car or home), and on the surface that seems fair enough, but is already fairly discriminatory. But it's far worse than that; if you have a bad credit score, you can't even rent an apartment. And many employers won't hire you. It's not just used to calculate what interest rate you get charged; it's used to judge your value as a human being in general.
- sct202 7y ago
  There's also a ChexSystem that banks use to block people from opening accounts, which is like basically blocking someone from even being able to get started doing anything financially since so much of it is digital now.
- mc32 7y ago
  So it’s almost like you’re saying it’s too coarse? Before FICO you either had to have lots of cash or you had to have people both of you knew vouch for you. FICO enables people to get credit with less hassle than the traditional pre-FICO method. I’d prefer FICO over s system which tries to manage my daily behavior and may publicly shame me for small transgressions. With that said, I hope we don’t allow a Chinese style social scoring system and I hope we see push back against this even if it’s not government mandated but a “voluntary choice” with a private entity.
  - caymanjim 7y ago
    My gripe isn't with credit history being used in the issuance of credit; that makes sense, and has made it a lot easier to borrow money. It also removes the discrimination that existed when bankers would simply refuse to extend credit to minorities. My point, as it relates to the article, is that what started out as a means of gauging financial creditworthiness is increasingly used as a means to gauge your worth in other aspects of life.
  - magduf 7y ago
    The FICO system isn't technically government-mandated, but it might as well be. If you try to get a security clearance, for instance, your FICO score/credit history is easily the #1 thing they look at.
deleted 7y ago
[deleted]
eerrisfs 7y ago
This makes me wonder which company made this "Social Credit Drone Strike List" [1] > In 2014, former CIA and NSA director Michael Hayden said in a public debate, “We kill people based on metadata.” > According to multiple reports and leaks, death-by-metadata could be triggered, without even knowing the target’s name, if too many derogatory checks appear on their profile. “Armed military aged males” exhibiting suspicious behavior in the wrong place can become targets, as can someone “seen to be giving out orders.” Such mathematics-based assassinations have come to be known as “signature strikes.” 1. rollingstone.com/politics/politics-features/how-to-survive-americas-kill-list-699334/
- freeflight 7y ago
  > Such mathematics-based assassinations have come to be known as “signature strikes.” Meet the algo behind the math assassinations, its name is literally SKYNET [0] [0] https://arstechnica.com/information-technology/2016/02/the-nsas-skynet-program-may-be-killing-thousands-of-innocent-people/ https://arstechnica.com/information-technology/2016/02/the-n...
mAEStro-paNDa 7y ago
Honest discussions and reporting around anything involving China is quite difficult to come by these days. I would very much recommend everyone take any Western reporting with a grain of salt, especially from obvious pro-U.S. sources. Much of the coverage of the situation in HK can be seen as evidence of this. On this topic specifically, Wired's recent article over this was a breath of fresh air[1] for the overall state of news reporting when it comes to China. Some clarification: First, when I talk about this slant in reporting Chinese affairs, it's not from a pro-CCP or pro-China position. Discussions about Chinese media and bias are still essential to have, but has nothing to do with the point I'm making here. Next, this issue isn't limited to just US media. There is arguably an observable bias even when it comes to Western academics that study or cover China in some capacity. I'm sure aspects of this system in China earns a healthy dose of criticism and skepticism. However, it's important to consider the way this may be reported in the West, especially as tensions heat up between China and the US. Just think, for example, that it would not be very difficult to cover the US's credit score system as authoritarian, racist, or Orwellian. In fact, such cases have been made in the past and have some weight to them. Just a thought. [1] https://www.wired.com/story/china-social-credit-score-system/ https://www.wired.com/story/china-social-credit-score-system...
- Aunche 7y ago
  > Wired's recent article over this was a breath of fresh air. I agree that this article is pretty good. Ironically, Wired's first article about China's social credit system generated a lot of misinformation about the social credit system in the first place. Nothing was factually incorrect, but most of the article was pure speculation and then everyone on the internet treated that as fact.
- xster 7y ago
  Big +1. This sounds hyperbolic but I really do think that our best chance for world peace is resting on us reading primary sources. If there are editorials, fine, but don't just confirmation bias and move on. Read the primary source.
dogman1233 7y ago
Fully, fully agree. Tech has opened up "channels of analysis" that the US spent a huge amount of time, effort, and social strife legislating against. Because of the hub-and-spoke model of tech and data, a lot of these channels have been re-opened in indirect, but socially important ways. What goes in one end for social media, comes out the other end in your insurance rates. How do we think Rocket Mortgage generates am instantaneous rate, when mortgage lenders used to rely on a great deal of relationship management and building to do the same loan issuance (and, they had credit scores back then, so it's not only an API into FICO that's changed). There are so many unknown unknowns here now. Previously, your mortgage rate bumped up if you lived in a red-line neighborhood. This was legislated against. Now, is the same thing happening with online banking/online health insurance rate quotes, etc., if you have a history of social media locations that place you in minority neighborhoods? Odds are, I bet yes, or something very similar. The more people that realize what is going into seemingly innocuous uses of tech - game apps, social media, food ordering, comes out in the other end in things that really matter to us - banking (this guy's kid spends $1k a month on candy crush), insurance (see the article), politics (we all know this), the more this can start to be legislated safely, and at least make the consumer aware.
zelon88 7y ago
> That Instagram pic showing you teasing a grizzly bear at Yellowstone with a martini in one hand, a bucket of cheese fries in the other, and a cigarette in your mouth, could cost you. On the other hand, a Facebook post showing you doing yoga might save you money. It will be a cold day in hell when any insurance company investigates you for potential fraud, decides you're not guilty, and then decides to LOWER your premiums as a result of their investigation. Premiums don't work that way. It's frustrating to see that float around as even a possibility from a blog like FastCompany that claims to know how the business world works. I find it incredibly frustrating anyone would even insinuate that insurance companies have a conscience.
- creaghpatr 7y ago
  If it's like car insurance, a competitor would use that data to lure you away with a lower premium.
  - zelon88 7y ago
    Except we know that's not how the younger generations buy insurance. [1] Many people buy the insurance they can afford. That means lowest up-front costs or flat out whoever will actually give you a policy. There's not nearly as much choice in the matter as you're making it out to be. Most people don't have the luxury of shopping for the best value plan. So the lowest barrier for picking up an insurance plan isn't who has the best price. It's who has the lowest standards for what they'll insure. The nature of the game means that those two selling points (price & availability) are mutually exclusive. [1] https://www.agencynation.com/millennials-independent-agents/ https://www.agencynation.com/millennials-independent-agents/
- pfranz 7y ago
  Insurance companies already give away FitBits, have "health fairs," I remember getting a $50 Visa gift card for participating in some health thing at work. These are all in exchange for lower premiums your employer is likely negotiated. I've heard similar stories about car insurance companies asking to install GPS into your car for discounts or per-mile billing. I'm not quite sure about the end-game for insurance companies for things that aren't about catching fraud. If we have perfect data then we lose the risk-pooling for insurance--then you're no longer insuring against anything and that seems like a different type of business to me.
olivermarks 7y ago
This could and should be a huge US election issue. Tulsi Gabbard is suing Google and has made some pretty strong statements about big tech. I fear all the other candidates are under the sway of the DC lobbyists where Google. FB et al spend tens of millions
- olivermarks 7y ago
  https://www.bloomberg.com/news/articles/2019-01-22/google-set-2018-lobbying-record-as-washington-techlash-expands https://www.bloomberg.com/news/articles/2019-01-22/google-se...
squirrelicus 7y ago
Adding reasons why opting out of social media 10 years ago was and remains a good idea.
bassman9000 7y ago
As long as it's not state enforced, not equivalent.
Nasrudith 7y ago
Comparing it to social credit is disingenuous even though there are real problems with existing and proposed ratings and their applications. Social Credit is a wrong-headed authoritarian tool of control that is enforced and has jack and shit to do with actual creditworthiness. It only "works" from being pressures because otherwise the companies shouldn't give a shit if they want to optimize for profits. If you tried to sell a loan evaluation system for banks based upon how they treated their parents in the US or Europe they would tell you to not waste their time again. There are problems with the current scoring systems of course - the burden of proof for identity theft is utterly backwards, credit scores have major "how good of a cash cow are you" aspects mixed in to what should be pure reliability, and idiots in recruiting use it for employement evaluation when it is utterly irrelevant. Even if the system winds up unjust and stupid there are large differences - the comparison isn't helpful.
mostvexing 7y ago
These false equivalences are not helpful at all.
- pertymcpert 7y ago
  Why are they false?
JohnFen 7y ago
Indeed so. That is one of the major reasons why I don't use social media, and go to great lengths to evade the pervasive spying that major tech companies have brought into fashion.

127 points by neuro 7y ago

In China, the government has the ability to make me disappear whereas a silicon valley corp. can't. That's the major difference here.

Finnucane 7y ago
True, they can just ruin your life, but at least they can't just kill you. That makes it totally okay.
- cameronbrown 7y ago
  That's at least something citizens can act upon. In China they'll just bury you.
  - anamexis 7y ago
    Just like citizens can act up on, say, getting blackballed by Google and having their email gone forever?
    - greenyoda 7y ago
      It's easy for citizens to fight back against that - by not using Google as their email provider (and educating others about the risks). There are many free and paid alternatives to Gmail.
    - vorpalhex 7y ago
      Are you comparing losing your email versus being kidnapped and held in jail without access to a lawyer or courts?
      - anamexis 7y ago
        No, I'm saying this is a thing that deeply impacts people's lives which they cannot, in fact, "act upon" when they are wronged.
        cameronbrown 7y ago
        Take regular backups has always been good advice, that doesn't go away just because a company handles your email for you. Always take archives from cloud services just like you backup HDDs, use a custom domain to forward inbound email traffic and you'll be 100% protected. You chose the worst example of Google being able to ruin somebody's life when if they really wanted to be evil, they'd have a monumental amount of data for blackmail, the ability to silence you online, etc..
dlphn___xyz 7y ago
just wait til the red flag law gets passed
- neonIcon 7y ago
  Careful there dlphn__xyz... I'd hate to have to report you /s
throwawaymanbot 7y ago
Still Not a valid reason to allow this abomination to happen here. You dont think that some Govt Dept wont strong arm this technology to do things to you and everyone who supports you?

Vaslo 7y ago

"The most disturbing attribute of a social credit system is not that it’s invasive, but that it’s extralegal. Crimes are punished outside the legal system, which means no presumption of innocence, no legal representation, no judge, no jury, and often no appeal. In other words, it’s an alternative legal system where the accused have fewer rights." Also, what if you disagree with me politically about illegal immigration - does that give you a "right" to ban me from your establishment? Imagine it on the other side as well for people who are against abortion.

astura 7y ago
Id imagine I wouldn't be welcome in a church or mosque because of my views on abortion.
- hprotagonist 7y ago
  you imagine incorrectly. it’s trivially easy to find faith communities with many views on that issue.
  - lliamander 7y ago
    Respectfully, there are faith communities that hold any number of views without regard to their consistency with founding documents or traditions of teachings. I heartily recommend that for anyone who is genuinely seeking spiritual guidance to avoid religious establishments that do not challenge present cultural norms and mores. However, it is very unlikely that a church is going to keep someone out, especially a non-believer, because of their views on abortion. Church attendance is generally open to everyone. It is participation in the sacraments (baptism, communion, etc.) and leadership that often requires doctrinal commitments.
    - hprotagonist 7y ago
      respectfully, that sounds like a fairly mealy-mouthed implicit endorsement of a very specific 19th century form of textual literalism to me. Speak plainly when you state a position, and I'll be happy to return the favor. If i'm reading you properly, you ought to know beforehand that we have probably very different hermeneutics. I would be happy to start on this issue by pointing to any number of long-running (like, millenia-long) ideas on this and other complex moral issues that are littered throughout the many texts of abrahamic faiths. Claiming that there is exactly one intepretation that's valid (which you implicitly do) is a crabbed and limiting way of reading and living pretty much all of the time. On this specific front, my church only recognizes two sacraments, and both are open to all. With regards to challenging present norms and mores, I am happy to agree, albeit in a way that you almost certainly don't mean. For instance: the more of the body of christ that locks themselves to concentration camp gates, the better!

mytailorisrich 7y ago

Insurances assessing risk based on publicly available information (social media posts) is in no way comparable to a social credit system. PatronScan looks potentially more dangerous, but the law in the US and UK (for example and afaik) is that you are free to refuse service to anyone you please as long as it is not illegal discrimination (e.g. in the UK, race or sexual orientation). Once again what this highlights is the power gained by these online platforms. On the one hand as private companies they have no obligation of universal service, on the other hand some of them have so much power that being excluded has a real impact on people. This reinforces my opinion that either these tech giants will effectively rule, or they will have to be controlled in a way similar to what China does in order to keep decisions on censorship, exclusion, and provision of service within public hands.

mieseratte 7y ago
> but the law in the US and UK (for example) is that you are free to refuse service to anyone you please as long as it is not illegal discrimination (e.g. in the UK, race or sexual orientation). There is more to discrimination law, including "gotchas" like adverse impact. > or they will have to be controlled in a way similar to what China does in order to keep decisions on censorship, exclusion, and provision of service within public hands. The Western nations seem more concerned with who and what is banned from a platform, the Chinese the opposite, banning speech the Party doesn't like and, perhaps more in line with the West, leveraging these companies as intelligence assets. In this way, the Silicon Valley companies themselves are more like the Chinese whereas the current administration less so.
- mytailorisrich 7y ago
  The point is that access to many platforms has become important enough that private companies should not be the ones calling the shots. This includes access to service and censorship. I fear that we have opened the pandora box and now the alternatives are those I mentioned. > There is more to discrimination law, including "gotchas" like adverse impact. Not in the UK, unless mandated by law in specific industries.

throwaway_law 7y ago

Isn't this already done to a great extent through black balling? SV recruiters blackball candidates/potential employees (often for no reason than petty vindictiveness); SV incubators/VCs use blackball lists; even insurance has internal lists in order to assess and deny claims based on nothing other than the "social credit" of the claimant rather than the merits of the claim itself. They even go a step further and insurance will rate/rank a claimant's attorney, so you may have a good claim that gets denied because you have a low ranking attorney, or you may have a weak claim they approve because your attorney is highly ranked (probably has a number of jury verdicts in similar cases).

mountainofdeath 7y ago
I have never heard of a blackball list at large SV companies unless it was something egregious that person did. Things like starting a fight with security, coming to an interview drunk, etc. will get you on it.
- thecleaner 7y ago
  >coming to an interview drunk, etc. will get you on it. Really what about the SV party culture ? And if the companies are that professional and only care about your work why does the age bias exist ?
  - TulliusCicero 7y ago
    > Really what about the SV party culture ? What party culture? I worked at Google in Mountain View and never saw any of this, nor "brogrammers", nor endless mandated/encouraged overtime, or most of the other stereotypes that seem to perpetuate themselves.
    - thecleaner 7y ago
      >I worked at Google in Mountain View and never saw any of this, nor "brogrammers"... How long did you work there >5 or >10 years or sth like that or just a stint ?

lidHanteyk 7y ago

Didn't this happen before with some group called "Klout" or some such? I don't remember what happened to them.

the_watcher 7y ago
Purchased by Lithium Technologies in 2014, shuttered in 2018. [0] [0] https://en.wikipedia.org/wiki/Klout https://en.wikipedia.org/wiki/Klout
freehunter 7y ago
They were bought for their machine learning algorithms and the purchaser ditched the rating service because they recognized it was a bad idea.

andrerm 7y ago

Why do you think Google is indexing all your online purchases? Look, online tracking inevitably leads to behavioral profiling. Put two behavioral profiles side by side and you have scoring, ranking. Add some weight on behaviors, age, income etc. and you have a social credit system. Edit: the question is not if big tech already have social scores in us or not. The questions are when and how they will use it

word-reader 7y ago
Exactly; let's say you have a social media record or even a browsing history that indicates labor activism (or drug use, or health issues, or any other kind of activism). That data is saved somewhere, associated with you directly or just assembled into a behavioral profile for ad targeting purposes -- at least for now and as far as we can tell. And like racial bias, this data can be dumped into a neural net with a score at the end to launder what exactly is going on.
goatinaboat 7y ago
Why do you think Google is indexing all your online purchases? See https://www.indiatimes.com/technology/news/google-scans-your-gmail-for-invoices-receipts-to-track-what-you-by-online-367598.html https://www.indiatimes.com/technology/news/google-scans-your...

mountainofdeath 7y ago

Fundamentally, for a service to not be discriminatory, it must be regulated like a utility (e.g. telephone, electric, etc). Using something that has the same functionality as a utility but is not regulated as such raises some questions. Also, profiling social media is nothing new. It's not uncommon for insurance companies to hire private investigators to look into suspicious cases. One case I heard was a man going on disability citing being homebound but the private investigator found evidence completely to the contrary. I fail to see how this is any different.

caymanjim 7y ago

Your FICO score and detailed credit history already serve this purpose to a large degree. Everyone knows that affects whether or not you can get a loan (and thus purchase a car or home), and on the surface that seems fair enough, but is already fairly discriminatory. But it's far worse than that; if you have a bad credit score, you can't even rent an apartment. And many employers won't hire you. It's not just used to calculate what interest rate you get charged; it's used to judge your value as a human being in general.

sct202 7y ago
There's also a ChexSystem that banks use to block people from opening accounts, which is like basically blocking someone from even being able to get started doing anything financially since so much of it is digital now.
mc32 7y ago
So it’s almost like you’re saying it’s too coarse? Before FICO you either had to have lots of cash or you had to have people both of you knew vouch for you. FICO enables people to get credit with less hassle than the traditional pre-FICO method. I’d prefer FICO over s system which tries to manage my daily behavior and may publicly shame me for small transgressions. With that said, I hope we don’t allow a Chinese style social scoring system and I hope we see push back against this even if it’s not government mandated but a “voluntary choice” with a private entity.
- caymanjim 7y ago
  My gripe isn't with credit history being used in the issuance of credit; that makes sense, and has made it a lot easier to borrow money. It also removes the discrimination that existed when bankers would simply refuse to extend credit to minorities. My point, as it relates to the article, is that what started out as a means of gauging financial creditworthiness is increasingly used as a means to gauge your worth in other aspects of life.
- magduf 7y ago
  The FICO system isn't technically government-mandated, but it might as well be. If you try to get a security clearance, for instance, your FICO score/credit history is easily the #1 thing they look at.

deleted 7y ago

[deleted]

eerrisfs 7y ago

This makes me wonder which company made this "Social Credit Drone Strike List" [1] > In 2014, former CIA and NSA director Michael Hayden said in a public debate, “We kill people based on metadata.” > According to multiple reports and leaks, death-by-metadata could be triggered, without even knowing the target’s name, if too many derogatory checks appear on their profile. “Armed military aged males” exhibiting suspicious behavior in the wrong place can become targets, as can someone “seen to be giving out orders.” Such mathematics-based assassinations have come to be known as “signature strikes.” 1. rollingstone.com/politics/politics-features/how-to-survive-americas-kill-list-699334/

freeflight 7y ago
> Such mathematics-based assassinations have come to be known as “signature strikes.” Meet the algo behind the math assassinations, its name is literally SKYNET [0] [0] https://arstechnica.com/information-technology/2016/02/the-nsas-skynet-program-may-be-killing-thousands-of-innocent-people/ https://arstechnica.com/information-technology/2016/02/the-n...

mAEStro-paNDa 7y ago

Honest discussions and reporting around anything involving China is quite difficult to come by these days. I would very much recommend everyone take any Western reporting with a grain of salt, especially from obvious pro-U.S. sources. Much of the coverage of the situation in HK can be seen as evidence of this. On this topic specifically, Wired's recent article over this was a breath of fresh air[1] for the overall state of news reporting when it comes to China. Some clarification: First, when I talk about this slant in reporting Chinese affairs, it's not from a pro-CCP or pro-China position. Discussions about Chinese media and bias are still essential to have, but has nothing to do with the point I'm making here. Next, this issue isn't limited to just US media. There is arguably an observable bias even when it comes to Western academics that study or cover China in some capacity. I'm sure aspects of this system in China earns a healthy dose of criticism and skepticism. However, it's important to consider the way this may be reported in the West, especially as tensions heat up between China and the US. Just think, for example, that it would not be very difficult to cover the US's credit score system as authoritarian, racist, or Orwellian. In fact, such cases have been made in the past and have some weight to them. Just a thought. [1] https://www.wired.com/story/china-social-credit-score-system/ https://www.wired.com/story/china-social-credit-score-system...

Aunche 7y ago
> Wired's recent article over this was a breath of fresh air. I agree that this article is pretty good. Ironically, Wired's first article about China's social credit system generated a lot of misinformation about the social credit system in the first place. Nothing was factually incorrect, but most of the article was pure speculation and then everyone on the internet treated that as fact.
xster 7y ago
Big +1. This sounds hyperbolic but I really do think that our best chance for world peace is resting on us reading primary sources. If there are editorials, fine, but don't just confirmation bias and move on. Read the primary source.

dogman1233 7y ago

Fully, fully agree. Tech has opened up "channels of analysis" that the US spent a huge amount of time, effort, and social strife legislating against. Because of the hub-and-spoke model of tech and data, a lot of these channels have been re-opened in indirect, but socially important ways. What goes in one end for social media, comes out the other end in your insurance rates. How do we think Rocket Mortgage generates am instantaneous rate, when mortgage lenders used to rely on a great deal of relationship management and building to do the same loan issuance (and, they had credit scores back then, so it's not only an API into FICO that's changed). There are so many unknown unknowns here now. Previously, your mortgage rate bumped up if you lived in a red-line neighborhood. This was legislated against. Now, is the same thing happening with online banking/online health insurance rate quotes, etc., if you have a history of social media locations that place you in minority neighborhoods? Odds are, I bet yes, or something very similar. The more people that realize what is going into seemingly innocuous uses of tech - game apps, social media, food ordering, comes out in the other end in things that really matter to us - banking (this guy's kid spends $1k a month on candy crush), insurance (see the article), politics (we all know this), the more this can start to be legislated safely, and at least make the consumer aware.

zelon88 7y ago

> That Instagram pic showing you teasing a grizzly bear at Yellowstone with a martini in one hand, a bucket of cheese fries in the other, and a cigarette in your mouth, could cost you. On the other hand, a Facebook post showing you doing yoga might save you money. It will be a cold day in hell when any insurance company investigates you for potential fraud, decides you're not guilty, and then decides to LOWER your premiums as a result of their investigation. Premiums don't work that way. It's frustrating to see that float around as even a possibility from a blog like FastCompany that claims to know how the business world works. I find it incredibly frustrating anyone would even insinuate that insurance companies have a conscience.

creaghpatr 7y ago
If it's like car insurance, a competitor would use that data to lure you away with a lower premium.
- zelon88 7y ago
  Except we know that's not how the younger generations buy insurance. [1] Many people buy the insurance they can afford. That means lowest up-front costs or flat out whoever will actually give you a policy. There's not nearly as much choice in the matter as you're making it out to be. Most people don't have the luxury of shopping for the best value plan. So the lowest barrier for picking up an insurance plan isn't who has the best price. It's who has the lowest standards for what they'll insure. The nature of the game means that those two selling points (price & availability) are mutually exclusive. [1] https://www.agencynation.com/millennials-independent-agents/ https://www.agencynation.com/millennials-independent-agents/
pfranz 7y ago
Insurance companies already give away FitBits, have "health fairs," I remember getting a $50 Visa gift card for participating in some health thing at work. These are all in exchange for lower premiums your employer is likely negotiated. I've heard similar stories about car insurance companies asking to install GPS into your car for discounts or per-mile billing. I'm not quite sure about the end-game for insurance companies for things that aren't about catching fraud. If we have perfect data then we lose the risk-pooling for insurance--then you're no longer insuring against anything and that seems like a different type of business to me.

olivermarks 7y ago

This could and should be a huge US election issue. Tulsi Gabbard is suing Google and has made some pretty strong statements about big tech. I fear all the other candidates are under the sway of the DC lobbyists where Google. FB et al spend tens of millions

olivermarks 7y ago
https://www.bloomberg.com/news/articles/2019-01-22/google-set-2018-lobbying-record-as-washington-techlash-expands https://www.bloomberg.com/news/articles/2019-01-22/google-se...

squirrelicus 7y ago

Adding reasons why opting out of social media 10 years ago was and remains a good idea.

bassman9000 7y ago

As long as it's not state enforced, not equivalent.

Nasrudith 7y ago

Comparing it to social credit is disingenuous even though there are real problems with existing and proposed ratings and their applications. Social Credit is a wrong-headed authoritarian tool of control that is enforced and has jack and shit to do with actual creditworthiness. It only "works" from being pressures because otherwise the companies shouldn't give a shit if they want to optimize for profits. If you tried to sell a loan evaluation system for banks based upon how they treated their parents in the US or Europe they would tell you to not waste their time again. There are problems with the current scoring systems of course - the burden of proof for identity theft is utterly backwards, credit scores have major "how good of a cash cow are you" aspects mixed in to what should be pure reliability, and idiots in recruiting use it for employement evaluation when it is utterly irrelevant. Even if the system winds up unjust and stupid there are large differences - the comparison isn't helpful.

mostvexing 7y ago

These false equivalences are not helpful at all.

pertymcpert 7y ago
Why are they false?

JohnFen 7y ago

Indeed so. That is one of the major reasons why I don't use social media, and go to great lengths to evade the pervasive spying that major tech companies have brought into fashion.