10 ms·

I always find it very strange that we don't apply spam filters and recent machine learning techniques to the problem of filtering comments. If I looked at my e

I always find it very strange that we don't apply spam filters and recent machine learning techniques to the problem of filtering comments. If I looked at my email unfiltered, I'd think one in three of the people sending me messages in the past day were unusually concerned about the state of my penis. From a not-adding-anything-to-the-discussion perspective, dead baby pictures and random threats of rape or acts of violence are even less useful than those adds telling me how to work at home. From a cost-benefit perspective for the people involved, it's very easy for someone relatively unimportant to get an outsized influence on social interactions if they can easily reach people who actually matter with harassing comments, at virtually no cost to themselves. So why don't we use tools to shape the online discussion and feedback mechanisms, the same way we do with our email inboxes? Perhaps I'm just too out of the loop to see it being done.

eli 12y ago
Maybe there's some value there, but my experience is that you can rarely solve a "people" problem with a purely technical solution. Also, as a society, I'd like to think we can do better than "sorry about the rape/murder threats, we'll try to hide most of them from you"
- cyorir 12y ago
  Agreed. Applying filters hides the symptoms without treating the underlying illness, which is a society that is infested with sexism. We can treat the symptoms by filtering but at the end of the day we need strategies for making society less sexist.
  - autokad 12y ago
    get over yourself. you should see the things people say to me in facebook chess let alone playing starcraft. its not because im a male or female, its because people are assholes but we dont need some sort of mind police to 'solve the problem'.
    - cyorir 12y ago
      Where do I argue in favor of mind police? Don't put words in my mouth for me, thank you. I look at how attitudes towards things like race have changed in the past 50 years. That is not a result of regulation imposed on society, that is a result of changing attitudes and culture in society. IMHO the same thing ought to happen with respects to sexism, we should hope that the attitudes and culture of society as a whole begin to shift.
  - NotAtWork 12y ago
    > Applying filters hides the symptoms without treating the underlying illness, which is a society that is infested with sexism. This doesn't follow from the symptoms being cited, which is that a small subset of people say incredibly offensive and inappropriate things online. If a million people see a video, and 1% of the people respond, and 3% of the people responding are crazy, that's 300 crazy people responding. If each crazy person sends 15 emails before finding a new thing to obsess about, that's 4,500 threatening or otherwise crazy emails in response - which certainly seems like a flood to the person being responded to - but doesn't actually tell us anything about the population watching the video, at large, except that it has a similar percentage of crazy people to society at large. Continuing the hypothetical: if you assume a normal person only sends 2 emails, then you only have 19000 normal emails to 4500 crazy ones, because crazy people are more talkative. At 1.2 emails for a normal person, It's 11000 to 4500, which makes crazy emails over a quarter of all messages received in response - a vastly disproportionate number considering they were 3% of respondents. One thing that automated filters can provide is sentiment analysis and looking at related messages, to pull out the underlying distribution of people responding (and hence seeing it's just people being people), rather than just seeing the surface level "We have 25% crazy emails!" I would have to see a much more detailed analysis of the emails, responses, and distribution of them to conclude that it's a widespread problem, rather than a small crazy contingent being amplified above their importance by just posting a lot.
    - cyorir 12y ago
      My apologies, I phrased things poorly. I don't want to say a majority of society is bad. I just think that even a small contingent, if loud enough, can make society seem more toxic than it needs to be.
      - seanflyon 12y ago
        Wouldn't making it harder for that small toxic contingent to be heard improve the situation?
        georgiapeach 12y ago
- NotAtWork 12y ago
  Changing the incentives of a situation, such as raising the effort required to get a harassing message through the filters, isn't a "purely technical solution" any more than most economics strategies are. I'd argue it's making use of technology to make a people oriented solution: people do less of things that are harder to do, and seeing less garbage even if you still see some makes the place seem more pleasant. > I'd like to think we can do better than "sorry about the rape/murder threats, we'll try to hide most of them from you" The best we can ever do, without swerving way out in to super repressive regimes, is impact the rate at which it happens, the response of other people, etc and not that it happens at all.
Torgo 12y ago
I would love to see an analysis on a site by site basis of what they consider trolls and what they consider highly-rated comments. I suspect however it would be found it would align more closely around in-group/out-group than content.
- tonypace 12y ago
  It seems likely, but the whole point of censoring extreme comments is to make those commenters an out group. Comment filtering would have edge cases, just like spam and porn filtering does. Do the dangers outweigh the value of pushing people who speak like this into an out group?
  - wutbrodo 12y ago
    I don't know how you interpreted Torgo's comment as worrying that inappropriate/extreme comments would be treated as an out-group. He/she said that he/she suspects that human-rating of comments would be based more on their (dis)agreement with the popular view (particularly on controversial topics) as opposed to the extremeness/inappropriateness of the content. There's a massive difference between a comment being 100% civil in their dissent, as in "I'm not convinced that you can attribute those causes to that effect, given XYZ" and "DIE YOU ASSHOLE YOU'RE WRONG I'LL KILL YOU". Torgo's suspicion is that filtering would be done based on disagreement vs objectively offensive/uncivil content.
    - tonypace 12y ago
      My point is that a comment filtering system is consciously based on the identification and exclusion of a group, which we'll call the out group. Of course it's limits will be debatable, just as the definition of spam is debatable. If what you're excluding is political speech of a sort, then someone is going to be able to accuse the system of censorship, no matter where the line is drawn. It's inherent in the system's goals. Personally, I think that's a point where any system is going to be rightfully challenged and examined critically, but I think it's still worth doing on a greater good basis. Spam filtering works, it makes my inbox a more tolerable place. It's a good idea to extend it to other types of crappy speech that makes other peoples' lives worse.

eli 12y ago

Maybe there's some value there, but my experience is that you can rarely solve a "people" problem with a purely technical solution. Also, as a society, I'd like to think we can do better than "sorry about the rape/murder threats, we'll try to hide most of them from you"

cyorir 12y ago
Agreed. Applying filters hides the symptoms without treating the underlying illness, which is a society that is infested with sexism. We can treat the symptoms by filtering but at the end of the day we need strategies for making society less sexist.
- autokad 12y ago
  get over yourself. you should see the things people say to me in facebook chess let alone playing starcraft. its not because im a male or female, its because people are assholes but we dont need some sort of mind police to 'solve the problem'.
  - cyorir 12y ago
    Where do I argue in favor of mind police? Don't put words in my mouth for me, thank you. I look at how attitudes towards things like race have changed in the past 50 years. That is not a result of regulation imposed on society, that is a result of changing attitudes and culture in society. IMHO the same thing ought to happen with respects to sexism, we should hope that the attitudes and culture of society as a whole begin to shift.
- NotAtWork 12y ago
  > Applying filters hides the symptoms without treating the underlying illness, which is a society that is infested with sexism. This doesn't follow from the symptoms being cited, which is that a small subset of people say incredibly offensive and inappropriate things online. If a million people see a video, and 1% of the people respond, and 3% of the people responding are crazy, that's 300 crazy people responding. If each crazy person sends 15 emails before finding a new thing to obsess about, that's 4,500 threatening or otherwise crazy emails in response - which certainly seems like a flood to the person being responded to - but doesn't actually tell us anything about the population watching the video, at large, except that it has a similar percentage of crazy people to society at large. Continuing the hypothetical: if you assume a normal person only sends 2 emails, then you only have 19000 normal emails to 4500 crazy ones, because crazy people are more talkative. At 1.2 emails for a normal person, It's 11000 to 4500, which makes crazy emails over a quarter of all messages received in response - a vastly disproportionate number considering they were 3% of respondents. One thing that automated filters can provide is sentiment analysis and looking at related messages, to pull out the underlying distribution of people responding (and hence seeing it's just people being people), rather than just seeing the surface level "We have 25% crazy emails!" I would have to see a much more detailed analysis of the emails, responses, and distribution of them to conclude that it's a widespread problem, rather than a small crazy contingent being amplified above their importance by just posting a lot.
  - cyorir 12y ago
    My apologies, I phrased things poorly. I don't want to say a majority of society is bad. I just think that even a small contingent, if loud enough, can make society seem more toxic than it needs to be.
    - seanflyon 12y ago
      Wouldn't making it harder for that small toxic contingent to be heard improve the situation?
      - georgiapeach 12y ago
NotAtWork 12y ago
Changing the incentives of a situation, such as raising the effort required to get a harassing message through the filters, isn't a "purely technical solution" any more than most economics strategies are. I'd argue it's making use of technology to make a people oriented solution: people do less of things that are harder to do, and seeing less garbage even if you still see some makes the place seem more pleasant. > I'd like to think we can do better than "sorry about the rape/murder threats, we'll try to hide most of them from you" The best we can ever do, without swerving way out in to super repressive regimes, is impact the rate at which it happens, the response of other people, etc and not that it happens at all.

Torgo 12y ago

I would love to see an analysis on a site by site basis of what they consider trolls and what they consider highly-rated comments. I suspect however it would be found it would align more closely around in-group/out-group than content.

tonypace 12y ago
It seems likely, but the whole point of censoring extreme comments is to make those commenters an out group. Comment filtering would have edge cases, just like spam and porn filtering does. Do the dangers outweigh the value of pushing people who speak like this into an out group?
- wutbrodo 12y ago
  I don't know how you interpreted Torgo's comment as worrying that inappropriate/extreme comments would be treated as an out-group. He/she said that he/she suspects that human-rating of comments would be based more on their (dis)agreement with the popular view (particularly on controversial topics) as opposed to the extremeness/inappropriateness of the content. There's a massive difference between a comment being 100% civil in their dissent, as in "I'm not convinced that you can attribute those causes to that effect, given XYZ" and "DIE YOU ASSHOLE YOU'RE WRONG I'LL KILL YOU". Torgo's suspicion is that filtering would be done based on disagreement vs objectively offensive/uncivil content.
  - tonypace 12y ago
    My point is that a comment filtering system is consciously based on the identification and exclusion of a group, which we'll call the out group. Of course it's limits will be debatable, just as the definition of spam is debatable. If what you're excluding is political speech of a sort, then someone is going to be able to accuse the system of censorship, no matter where the line is drawn. It's inherent in the system's goals. Personally, I think that's a point where any system is going to be rightfully challenged and examined critically, but I think it's still worth doing on a greater good basis. Spam filtering works, it makes my inbox a more tolerable place. It's a good idea to extend it to other types of crappy speech that makes other peoples' lives worse.