12 ms·

Decentralised content moderation

113 points by AstroNoise58 6y ago

AstroNoise58 6y ago
I find it pretty interesting that Martin does not mention the kind of community member-driven up/downvote mechanism found on this site (and elsewhere) as an example of decentralised content moderation. Edit: now I see Slashdot and Reddit mentioned at the end in the updates section (I don't remember seeing them on my first read, but that might just be me).
- meheleventyone 6y ago
  Isn't it just an example of democratic content moderation? We up vote, down vote and flag content. We don't get the ability to do so unless we are a community member of some tenure. It's augmented by centralized moderation by a handful of moderators. How well it works is always a topic here.
  - freeqaz 6y ago
    We don't see all of the countless hours spent by mods like dang to keep the quality high. It's a thankless job most of the time!
    - meheleventyone 6y ago
      Having moderated some large forums in the past I know!
  - randompwd 6y ago
    > Isn't it just an example of democratic content moderation A democracy makes great efforts to ensure 1 person = 1 vote. Online platforms do not.
    - deleted 6y ago
      [deleted]
- commandlinefan 6y ago
  FTA: "even though such filtering saves you from having to see things you don’t like, it doesn’t stop the objectionable content from existing". He doesn't want upvoting/downvoting, he wants complete eradication (of whatever the majority happens to objects to right now).
  - throwaway2245 6y ago
    > of whatever the majority happens to objects to right now I don't see that Martin Kleppmann is using 'democracy' to mean 'majoritarianism' here. He makes considered points about how to form and implement policies against harmful content, and appears to talk about agreement by consensus. Democracy and majoritarianism are (in general) quite different things. This might be more apparent in European democracies.
    - Viliam1234 6y ago
      He plays a little trick by saying "ultimately it should be the users themselves who decide what is acceptable or not". This has two meanings, somewhat contradictory. The straightforward meaning is that ultimately I decide what is acceptable or not for me, and you decide what is acceptable or not for you. We can, and likely will, have a different opinion on different things. But the following talk of "governance" and "democratic control" suggest that the one who ultimately decides are not users as individuals, but rather some kind of process that would be called democratic in some sense. Ultimately, someone else will make the decision for you... but you can participate in the process, if you wish... but if your opinions are too unusual, you will probably lose anyway... and then the rest of us will smugly congratulate ourselves for giving you the chance. > Democracy and majoritarianism are (in general) quite different things. Sure, a minority can have rights as long as it is popular, rich, well organized, able to make coalitions with other minorities, or too unimportant to attract anyone's attention. But that still means living under the potential threat. I don't see a reason why online communities would have to be built like this, if instead you could create a separate little virtual universe for everyone who wished to be left alone... and then invent good tools for navigating these universes, to make it convenient, from user perspective, to create their places, to invite and be invited, and to exlude those who don't follow the local rules (who in turn can create their own places and compete for popularity).
- maurys 6y ago
  He mentions Reddit at the end of the article, which is close enough in mechanism to Hackernews.
- Steltek 6y ago
  How would up/down votes work on a decentralized platform? Wouldn't it be easy to game by standing up your own server and wishing up a legion of sockpuppets? There's a whole Moonshot of spam resistance that's going to need to happen in Mastodon/Matrix/Whatever.
  - nine_k 6y ago
    Decentralized networks need trust, and trust is not a Boolean value. With a centralized service, trust is simple: how much you trust the single entity that represents the service. In a distributed network, nodes need to build trust to each other. In the best-known federated network, email, domain reputation is a thing. Various blacklists and graylists pass around trust values in bulk. So a node with a ton of sock puppets trying to spam votes (or content) is going to lose the trust of its peers fast, so the spam from it will end up marked as such. A well-run node will gain considerable trust with time. This, of course, while helpful, does not guarantee "fairness" of any kind. If technology and people's values clash, the values prevail. You cannot alter values with technology alone (even weapon technology).
  - dboreham 6y ago
    The problem you describe is called Sybil resistance and is known to be hard, but there are some example working systems such as Bitcoin.
- mytailorisrich 6y ago
  Up/downvote mechanisms always end up as agree/disagree votes. Moderation is not the same. It is not about agreeing but curating content that is not acceptable (off-topic, illegal, insulting). Article quote: "In decentralised social media, I believe that ultimately it should be the users themselves who decide what is acceptable or not" In my view that is only workable if that means users define the rules because, as said above, I think 'voting' on individual piece of content always leads to echo chambers and to censoring dissenting views. Of course this may be fine if within an online community focus on one topic or interest, but probably not if you want to foster open discussions and a plurality of views and opinions. We can observe this right here on HN. On submissions that are prone to trigger strong opinions downvotes and flagging explode.
- totemandtoken 6y ago
  "We vote on values, we bet on beliefs" - Robin Hanson. Voting tells us what we value but that doesn't mean what is good for us. It also treats all content as somewhat equivalent, which isn't true. A call to (maybe violent) action isn't the same thing as sharing a cute cat video.
nathias 6y ago
There is absolutely no difficulty, if you don't want censorship, just have a button that hides content you personally don't want and leave that decision to all individual users. What others wish to see or not is not your decision to make, and if some of them are illegal that should be a job for the police not some ego-triping janny.
- tdjsnelling 6y ago
  Totally agree. Why should someone else I've never met decide what I can and can't see? Leave that decision up to the individual user, and they can tailor their own experience as they desire. Allow them to filter out particular words, phrases, other users and so on.
halfmatthalfcat 6y ago
I've been thinking hard about decentralized content moderation, especially around chatrooms, for years. More specifically because I'm building a large, chatroom-like service for TV. I think it's evident from Facebook, Twitter, et al that human moderation of very dynamic situations is incredibly hard, maybe even impossible. I've been brewing up strategies of letting the community itself moderate because a machine really cannot "see" what content is good or bad, re: context. While I think that community moderation will inevitably lead to bubbles, it's a better and more organic tradeoff than letting a centralized service dictate what is and isn't "good".
- sandworm101 6y ago
  >> a centralized service dictate what is and isn't "good". This isn't movie reviews. Good and bad are not the standards. The standard is whether or not something is illegal. When the feds come knocking on your door because your servers are full of highly illegal content, "we let them moderate themselves" will be no defense.
  - pjc50 6y ago
    "Legal" is both local and can change over time. America has been in an unusual situation because it allows far more speech, especially speech right up to the boundary of "incitement to violence". However, the police have far too much to do, so in practice millions of blatently illegal death threats get sent every day and do not receive any police response. Hence the demand for a non-police response that can far more cheaply remove the death threats or threateners.
  - Mindwipe 6y ago
    "Legal" is not a very good determination. Discussion of homosexuality is "illegal" in many states. It is a moral imperative for systems to break those laws.
- eternalban 6y ago
  Make a cogent and non-authoritarian case for even having a "decentralized content moderation" which doesn't pass the "is this an oxymoron?" smell test. > I'm building a large, chatroom-like service for TV. So the profit motive is likely the motivation for applying a "central" doctrine of acceptable discourse using a decentralized mechanism. > While I think that community moderation will inevitably lead to bubbles Which allows for e.g. an athiest community to have content that rips religion x's scripture to shred (and why not?) in the same planet that also has a religion-x community that has content that takes a bat to over-reaching rationalism. Oh the horror! Diversity of thought. "We simply can not permit this."
lawrencevillain 6y ago
Obviously upvoting and downvoting is not enough for adequate moderation. There's still the aspect of people trolling and generally posting horrible things online. There's a reason Facebook had to pay $52 million to content moderators for the trauma/ptsd they suffered.
neiman 6y ago
We (at Almonit) work on a self-governing publication system which would bring democratic control to content moderation. We just wrote about its philosophy earlier this week. https://almonit.com/blog/2021-01-08/self-governing_internet_organizations_part_I.html https://almonit.com/blog/2021-01-08/self-governing_internet_...
- theylon 6y ago
  This is an interesting concept. Are there any implementations of this?
deleted 6y ago
[deleted]
emaro 6y ago
Matrix published an interesting concept for decentralised content moderation [0]. I think this is the way to go. Edit: Discussed here [1] and here [2]. [0]: https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix-without-backdoors https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix... [1]: https://news.ycombinator.com/item?id=24826951 https://news.ycombinator.com/item?id=24826951 [2]: https://news.ycombinator.com/item?id=24836987 https://news.ycombinator.com/item?id=24836987
jrexilius 6y ago
Good piece. This line articulates the problem well: "without objectivity and consistency, moderation can easily degenerate into a situation where one group of people forces their opinions on everyone else, like them or not." And it gets to the core of the problem. Objectivity and consistency are extremely difficult to scale and maintain over time. They require constant reinforcement from environment, context, and culture.
- zarkov99 6y ago
  The central problem here is in what circumstances free people should give up the right to decide what information they can consume. This is not a question that can be answered easily but without first accepting it as the central issue we are not going to make meaningful progress.
  - nindalf 6y ago
    That’s part of it. The other part is that free people say they don’t want to see illegal content (child exploitation imagery, terrorism, scams, sale of opioids etc). The platform needs to moderate to remove that. Then the same users also say they don’t want to see legal but distasteful (in their opinion) content like pornography, spam and so on. The platform then has to remove that as well. For most part platforms take decisions that will suit the majority of users.
    - Viliam1234 6y ago
      I don't want to see "legal but distasteful" content (and I would also add: annoying, boring, stupid, etc.)... and what I mean is that I don't want it to be shown to me... but I am okay if other people show it to each other. So instead of some global moderator (whether it be one person, or some complicated "democratic" process) deciding globally what is okay and what is not, I want many bubbles that can enforce their own rules, and the option to choose the bubbles I like. Then I will only be concerned about how to make the user interface as convenient as possible, so that the options are not only hypothetically there, but actually easy to use also by non-tech users.
EGreg 6y ago
Let's take one step back. Just like in the Title I vs Title II debate, let's go one step earlier. WHY do we have these issues in the first place? It's because our entire society is permeated with ideas about capitalism and competition being the best way to organize something, almost part of the moral fabric of the country. Someone "built it", now they ought to "own" the platform. Then they get all this responsibility to moderate, not moderate, or whatever. Compare with science, wikipedia, open source projects, etc. where things are peer reviewed before the wider public sees them, and there is collaboration instead of competition. People contribute to a growing snowball. There is no profit motive or market competition. There is no private ownership of ideas. There are no celebrities, no heroes. No one can tweet to 5 million people at 3 am. Somehow, this has mistakenly become a “freedom of speech” issue instead of an issue of capitalism and private ownership of the means of distribution. In this perverse sense, "freedom of speech" even means corporations should have a right to buy local news stations and tell news anchors the exact talking points to say, word for word, or replacing the human mouthpieces if they don't... Really this is just capitalism, where capital consists of audience/followers instead of money/dollars. Top down control by a corporation is normal in capitalism. You just see a landlord (Parler) crying about higher landlord ... ironically crying to the even higher landlord, the US government - to use force and “punish” Facebook. Going further, it means corporations (considered by some to have the same rights as people) using their infrastructure and distribution agreements to push messages and agendas crafted by a small group of people to millions. Celebrity culture is the result. Ashton Kutcher was the first to 1 million Twitter followers because kingmakers in the movie industry chose him earlier on to star in movies, and so on down the line. Many companies themselves employ social media managers to regularly moderate their own Facebook Pages and comments, deleting even off-topic comments. Why should they have an inalienable right to be on a platform? So inside their own website and page these private companies can moderate and choose not to partner with someone but private companies Facebook and Twitter should be prevented from making decisions about content on THEIR own platform. You want a platform that can’t kick you off? It’s called open source software, and decentralized networks. You know what they don’t have? Private ownership of the whole network. “But I built it so I get to own it” is the capitalist attitude that leads to exactly this situation. The only way we will get there is if people build it and then DON’T own the whole platform. Think about it!
jonathanstrange 6y ago
Since I'm using libp2p in Go for a side project, may I take this opportunity to ask how this could work in principle for a decentralized network? The way I see it, this seems to be impossible but maybe I'm missing something. For example, in my network, anyone can start a node and the user has full control over it. So how would you censor this node? The following ideas don't seem to work: 1. Voting or another social choice consensus mechanism. Problems: - Allows a colluding majority to mount DOS attacks against anyone. - Can easily be circumvented by changing host keys / creating a new identity. 2. The equivalent of a killfile: Users decide to blacklist a node, dropping all connections to it. Problems: - Easy to circumvent by creating new host keys / creating a new identity. 3. Karma system: This is just the same as voting / social choice aggregation and has the same problems. 4. IP banning by distributing the blocked IPs with the binaries in frequent updates. Problem: - Does not work well with dynamic IPs and VPNs. Basically, I can't see a way to prevent users from creating new identities / key pairs for themselves whenever the old one has been banned. Other than security by obscurity nonsense ("rootkit" on the user's machine, hidden keys embedded in binaries, etc.) or a centralized server as a gateway, how would you solve that problem?
- JulianMorrison 6y ago
  The way it works in Mastodon is that (1) not everyone runs a node, but there are many nodes, and they each have their own policies and can kick users off, and (2) nodes can blacklist other nodes they federate content from. This two level split allows node operators to think of most other users at the node level, which means dealing with far fewer entities. It provides users with a choice of hosts, but means that their choice has consequences.
- yorwba 6y ago
  > I can't see a way to prevent users from creating new identities / key pairs for themselves whenever the old one has been banned. You could prevent banned users from returning with a new identity by disallowing the creation of new identities. E.g. many Mastodon instances disable their signup pages and new users can only be added by the admins. If you don't want to put restrictions on new identities, you could still treat them as suspect by default. E.g. apply a kind of rate limiting where content created by new users is shown at most once per day and the limit rises slowly as the user's content is viewed more and more without requiring moderation. (This is a half-baked idea I had just now, so I'm sure there are many drawbacks. But it might be worth a shot.)
colllectorof 6y ago
It's pretty obvious that the tech crowd right now is so intoxicated by its own groupthink that these attempts to come up with "solutions" are going to have awful results. You don't even know what the problem really is. "I fear that many decentralised web projects are designed for censorship resistance not so much because they deliberately want to become hubs for neo-nazis, but rather out of a kind of naive utopian belief that more speech is always better. But I think we have learnt in the last decade that this is not the case." What you should have learned in the last decade is that social networks designed around virality, engagement and "influencing" are awful for the society in the long run. But somehow now the conversation has turned away from that and towards "better moderation". Engage your brain. Read Marshall McLuhan. The design of a medium is far more important than how it is moderated.
- crmrc114 6y ago
  "What you should have learned in the last decade is that social networks designed around virality, engagement and "influencing" are awful for the society in the long run." Yes, and don't forget the 24 hour news cycle with its focus on getting outrage and attention through fear. I did not know who Marshall McLuhan was until now- thanks for the tip!
  - ardy42 6y ago
    > Yes, and don't forget the 24 hour news cycle with its focus on getting outrage and attention through fear. Yeah, social media is just one in a series of possibly misguided techno-social "innovations," and it probably won't be the last. My understanding is that groups like the Amish don't reject technology outright, but adopt it selectively based on its effects on their society (and will even roll back things they've adopted if they're not working out). Wider society probably would benefit from a dose of that kind of wisdom right now, after decades of decades of "because we can"-driven "innovation."
    - jl2718 6y ago
      They also force all 17-year-olds to go live with and as ‘the English’ for two years and then decide whether they want to go back, and what things should be brought back with them.
jancsika 6y ago
> Censorship resistance means that anybody can say anything, without suffering consequences. I can't even get to the heart of the poster's argument. That's because the shitty state of all current social media software defines "anybody" as: * a single user making statements in earnest * a contractor tacitly working on behalf of some company * an employee or contractor working on behalf of a nation state * a botnet controlled by a company or nation state It's so bad that you can witness the failure in realtime on, say, Reddit. I'm sure I'm not the only one who has skimmed comments and thought, "Gee, that's a surprisingly reaction from lots of respondents." Then go back even 30 minutes later and the overwhelming reaction is now the opposite, with many comments in the interim about new or suspicious accounts and lots of moderation of the initial astroturfing effort. Those of us who have some idea of the scope of the problem (hopefully) become skeptical enough to resist rabbit-holes. But if you have no idea of the scope (or even the problem itself), you can easily get caught in a vicious cycle of being fed a diet of propaganda that is perhaps 80% outright fake news. As long as the state of the art remains this shitty (and there are plenty of monetary incentives for it to remain this way), what's the point of smearing that mendacity across a federated system?
gfodor 6y ago
You can cut off a large part of abuse by just creating a financial based incentive. Pay to gain access, and access can be revoked at which point you need to repay (perhaps on a progressive scale - ie the more you are banned, the harder it is to get back in.) Your identity confers a reputation level that influences filters so what you post is seen more often, so there is value in your account and you don't want to lose it. The SA forums did this and it helped immensely with keeping out spam (though not a silver bullet.) Any system where any rando can post any random thing with no gates is going to be much more of a slog to moderate than one where there are several gates that imply the person is acting in good faith.
eternalban 6y ago
I fear that many decentralised web projects are designed for censorship resistance not so much because they deliberately want to become hubs for neo-nazis, but rather out of a kind of naive utopian belief that more speech is always better. But I think we have learnt in the last decade that this is not the case. If we want technologies to help build the type of society that we want to live in, then certain abusive types of behaviour must be restricted. Thus, content moderation is needed. Let's unpack this: Axiom: a kind of naive utopian belief [exists that asserts] that more speech is always better. But I think we have learnt in the last decade that this is not the case. False premise. The "naive belief", based on the empirical evidence of history, is that prioritizing the supression of speech to address social issues is the hallmark of authoritarian systems. Martin also claims "we have learned" something that he is simply asserting as fact. My lesson from the last 3 decades has been that it was a huge mistake to let media ownership be concentrated in the hands of a few. We used to have laws against this in the 90s. Axiom: By "we" as in "we want", Martin means the community of likeminded people, aka the dreaded "filter bubble" or "community value system". Who is this "we", Martin? Theorem: If we want technologies to help build the type of society that we want to live in, then certain abusive types of behaviour must be restricted. We already see that the "we" of Martin is a restricted subset of "we the Humanity". There are "we" communities that disagree with Martin's on issues ranging from: the fundamental necessity for freedom of thougth and conscience; the positive value of diversity of thoughts; the positive value of unorthodox ("radical") thought; the fundamental identity of the concept of "community" with "shared values"; etc. Q.E.D.: Thus, content moderation is needed. Give the man a PhD. -- So here is a parable of a man named Donald Knuth. This Donald, while a highly respected and productive contributing member of the 'Community of Computer Scientists of America' [ACM, etc.], also sadly entertains irrational beliefs that "we" "know" to be superstitious non-sense. The reason that this otherwise sane man entertains this nonsensical thoughts is because of the "filter bubble" of the community he was raised in. Of course, to this day, Donald Knuth has never tried to force his views in the ACM on other ACM members, many of whom are devout athiests. And should Donald Knuth ever try to preach his religion in ACM, we would expect respectful but firm "community filter bubble" action of ACM telling Mr. Knuth to keep his religious views for his religious community. But, "[i]f we want technologies to help build the type of society that we want to live in" -- and my fellow "we", do "we" not agree that there is no room for Donald Knuth's religious nonsense in "our type of society"? -- would it not be wise to ensure that the tragedy that befell the otherwise thoughtful and rational Donald Knuth could happen to other poor unsuspecting people who happen to be born and raised in some "fringe" community? "Thus, content moderation is needed."
iamsb 6y ago
My suggestion is to not indulge in any content moderation which is not illegal. Only take down content which is required by a court order. Limit use of automated content moderation only for easy to solve cases like child pornography. Why? It is fairly clear at this point that content moderation at internet scale is not possible. Why? A. Using other users to flag dangerous content is not working. Which users do you trust to bestow this power with? How do remove this power from them? How do you control it becoming a digital lynch mob? Can you have users across political, gender, other dimensions All mostly not solvable problems. B. Is it possible to use machine learning? To some extent. But any machine learning algorithm will have inherent bias, because test data will also be produced by biased individuals. Also people will eventually figure out how to get around those algorithms as well. The causality between content published on the internet and action in real world is not immediate. It is not like someone is sitting in a crowded place and shouting fire causing a stampede. As there is a sufficient delay between speech and action, we can say that the medium the speech is published in is not the primary cause of the action, even if there is link. Chances of direct linkage are fairly rare and police/law should be able to deal with those. Content moderation, at least the way Twitter has been trying to do, has not been effective, created lot of ways for mobs to enforce censorship, and there is absolutely no real word positive impact of this censorship is. Only use of this moderation and censorship has been for right to claim victimhood and gain more viewer/readership to be honest.
- Pfhreak 6y ago
  You realize that the approach you suggest pushes out a different set of people, right? For example, a soldier with PTSD may want an environment that moderates content. Or a journalist with epilepsy may want a platform where people don't spam her with gifs designed to trigger epilepsy when she says something critical of a game release.
  - iamsb 6y ago
    I understand. Most of those can be achieved using privacy and sharing settings though and does not necessarily require content moderation.
JulianMorrison 6y ago
It's worth remembering that content moderation is an activity which causes people mental illness right now, because it's so unrelentingly awful. Attempts to decentralize this are going to be met with the problem that people don't want to be exposed to a stream of pedophilia, animal abuse, murderous racism and terrorist content and asked to score it for awfulness.
- totemandtoken 6y ago
  This point should be upvoted more. Decentralized moderation means decentralized pain. You can run standard philosophic thought experiments about "utility monsters" but at the end of the day a lot of online content causes real harm, not just to the direct victims but to the spectators. We'd need something like decentralized therapy in tandem with this type of moderation for it to even be considered remotely ethical, and even then I'm very skeptical.
- okokok___ 6y ago
  I agree, I think people underestimate the need to incentivize moderators. This is why I think some kind of cryptocurrency based solution to moderation is necessary.
  - JulianMorrison 6y ago
    Moderators don't just need incentives. They need therapy and support and monitoring and breaks and ideally, pre-vetting for strong mental health and no pre-existing bees in their bonnet. I worry that just making it a way to earn bitcoins risks it becoming one more way for poor people to scrape together pennies at the cost of giving themselves PTSD.
    - okokok___ 6y ago
      good point. It's almost like mechanical turk but for moderation.
kstrauser 6y ago
I'm active with Mastodon and absolutely love its moderation model. In a nutshell: - It's made up of a bunch of independent servers, or "instances". The common analogy here is to email systems. - If you want to join the federation, stand up an instance and start using it. Voila! Now you're part of it. - My instance has a lot of users, and I don't want to run them off, so it's in my own interest to moderate my own instance in a way that my community likes. Allow too much in without doing anything? They leave. Tighten it so that it starts losing its value? They leave. There's a feedback mechanism that guides me to the middle road. - But my users can leave for greener pastures if they think I'm doing a bad job and think another instance is better. They're not stuck with me. The end result is that there are thousands of instances with widely varied moderation policies. There are some "safe spaces" where people who've been sexually assaulted hang out and that have zero tolerance for harassment or trolling. There are others that are very laissez faire. There's a marketplace of styles to choose from, and no one server has to try to be a perfect fit for everyone. I realize that this is not helpful information for someone who wants to run a single large service. I bring it up just to point out that there's more than one way to skin that cat. (That final idiom would probably get me banned on some servers. And that's great! More power to that community for being willing and able to set policies, even if I wouldn't agree with them.)
- makeworld 6y ago
  Couldn't have said it better myself. Mastodon appears to solve all the moderation problems I've seen raised about social media.
MrXOR 6y ago
The problem of censorship on Twitter and other social media can be solved with a good moderator like dang (thank you!)
mikeveilleux 6y ago
Having been part of the email ecosystem where well established spam filtering and reputation management systems have been in place for years, I've found it interesting how close these more recent conversations are to those we've had around email messaging abuse. > Thus, as soon as a censorship-resistant social network becomes sufficiently popular, I expect that it will be filled with messages from spammers, neo-nazis, and child pornographers (or any other type of content that you consider despicable). Unfortunately, I agree this is likely the case, and also agree with many of the other points where there's unlikely to be an agreed upon approach at scale. I feel the two most important aspects of any moderation are transparency and consistency. I'd always like to know what community I'm joining. We'll likely see more niche communities continue to pop up on centralized and decentralized networks where the moderation, content and community can be more tailored to their own expectations.

113 points by AstroNoise58 6y ago

I find it pretty interesting that Martin does not mention the kind of community member-driven up/downvote mechanism found on this site (and elsewhere) as an example of decentralised content moderation. Edit: now I see Slashdot and Reddit mentioned at the end in the updates section (I don't remember seeing them on my first read, but that might just be me).

meheleventyone 6y ago
Isn't it just an example of democratic content moderation? We up vote, down vote and flag content. We don't get the ability to do so unless we are a community member of some tenure. It's augmented by centralized moderation by a handful of moderators. How well it works is always a topic here.
- freeqaz 6y ago
  We don't see all of the countless hours spent by mods like dang to keep the quality high. It's a thankless job most of the time!
  - meheleventyone 6y ago
    Having moderated some large forums in the past I know!
- randompwd 6y ago
  > Isn't it just an example of democratic content moderation A democracy makes great efforts to ensure 1 person = 1 vote. Online platforms do not.
  - deleted 6y ago
    [deleted]
commandlinefan 6y ago
FTA: "even though such filtering saves you from having to see things you don’t like, it doesn’t stop the objectionable content from existing". He doesn't want upvoting/downvoting, he wants complete eradication (of whatever the majority happens to objects to right now).
- throwaway2245 6y ago
  > of whatever the majority happens to objects to right now I don't see that Martin Kleppmann is using 'democracy' to mean 'majoritarianism' here. He makes considered points about how to form and implement policies against harmful content, and appears to talk about agreement by consensus. Democracy and majoritarianism are (in general) quite different things. This might be more apparent in European democracies.
  - Viliam1234 6y ago
    He plays a little trick by saying "ultimately it should be the users themselves who decide what is acceptable or not". This has two meanings, somewhat contradictory. The straightforward meaning is that ultimately I decide what is acceptable or not for me, and you decide what is acceptable or not for you. We can, and likely will, have a different opinion on different things. But the following talk of "governance" and "democratic control" suggest that the one who ultimately decides are not users as individuals, but rather some kind of process that would be called democratic in some sense. Ultimately, someone else will make the decision for you... but you can participate in the process, if you wish... but if your opinions are too unusual, you will probably lose anyway... and then the rest of us will smugly congratulate ourselves for giving you the chance. > Democracy and majoritarianism are (in general) quite different things. Sure, a minority can have rights as long as it is popular, rich, well organized, able to make coalitions with other minorities, or too unimportant to attract anyone's attention. But that still means living under the potential threat. I don't see a reason why online communities would have to be built like this, if instead you could create a separate little virtual universe for everyone who wished to be left alone... and then invent good tools for navigating these universes, to make it convenient, from user perspective, to create their places, to invite and be invited, and to exlude those who don't follow the local rules (who in turn can create their own places and compete for popularity).
maurys 6y ago
He mentions Reddit at the end of the article, which is close enough in mechanism to Hackernews.
Steltek 6y ago
How would up/down votes work on a decentralized platform? Wouldn't it be easy to game by standing up your own server and wishing up a legion of sockpuppets? There's a whole Moonshot of spam resistance that's going to need to happen in Mastodon/Matrix/Whatever.
- nine_k 6y ago
  Decentralized networks need trust, and trust is not a Boolean value. With a centralized service, trust is simple: how much you trust the single entity that represents the service. In a distributed network, nodes need to build trust to each other. In the best-known federated network, email, domain reputation is a thing. Various blacklists and graylists pass around trust values in bulk. So a node with a ton of sock puppets trying to spam votes (or content) is going to lose the trust of its peers fast, so the spam from it will end up marked as such. A well-run node will gain considerable trust with time. This, of course, while helpful, does not guarantee "fairness" of any kind. If technology and people's values clash, the values prevail. You cannot alter values with technology alone (even weapon technology).
- dboreham 6y ago
  The problem you describe is called Sybil resistance and is known to be hard, but there are some example working systems such as Bitcoin.
mytailorisrich 6y ago
Up/downvote mechanisms always end up as agree/disagree votes. Moderation is not the same. It is not about agreeing but curating content that is not acceptable (off-topic, illegal, insulting). Article quote: "In decentralised social media, I believe that ultimately it should be the users themselves who decide what is acceptable or not" In my view that is only workable if that means users define the rules because, as said above, I think 'voting' on individual piece of content always leads to echo chambers and to censoring dissenting views. Of course this may be fine if within an online community focus on one topic or interest, but probably not if you want to foster open discussions and a plurality of views and opinions. We can observe this right here on HN. On submissions that are prone to trigger strong opinions downvotes and flagging explode.
totemandtoken 6y ago
"We vote on values, we bet on beliefs" - Robin Hanson. Voting tells us what we value but that doesn't mean what is good for us. It also treats all content as somewhat equivalent, which isn't true. A call to (maybe violent) action isn't the same thing as sharing a cute cat video.

nathias 6y ago

There is absolutely no difficulty, if you don't want censorship, just have a button that hides content you personally don't want and leave that decision to all individual users. What others wish to see or not is not your decision to make, and if some of them are illegal that should be a job for the police not some ego-triping janny.

tdjsnelling 6y ago
Totally agree. Why should someone else I've never met decide what I can and can't see? Leave that decision up to the individual user, and they can tailor their own experience as they desire. Allow them to filter out particular words, phrases, other users and so on.

halfmatthalfcat 6y ago

I've been thinking hard about decentralized content moderation, especially around chatrooms, for years. More specifically because I'm building a large, chatroom-like service for TV. I think it's evident from Facebook, Twitter, et al that human moderation of very dynamic situations is incredibly hard, maybe even impossible. I've been brewing up strategies of letting the community itself moderate because a machine really cannot "see" what content is good or bad, re: context. While I think that community moderation will inevitably lead to bubbles, it's a better and more organic tradeoff than letting a centralized service dictate what is and isn't "good".

sandworm101 6y ago
>> a centralized service dictate what is and isn't "good". This isn't movie reviews. Good and bad are not the standards. The standard is whether or not something is illegal. When the feds come knocking on your door because your servers are full of highly illegal content, "we let them moderate themselves" will be no defense.
- pjc50 6y ago
  "Legal" is both local and can change over time. America has been in an unusual situation because it allows far more speech, especially speech right up to the boundary of "incitement to violence". However, the police have far too much to do, so in practice millions of blatently illegal death threats get sent every day and do not receive any police response. Hence the demand for a non-police response that can far more cheaply remove the death threats or threateners.
- Mindwipe 6y ago
  "Legal" is not a very good determination. Discussion of homosexuality is "illegal" in many states. It is a moral imperative for systems to break those laws.
eternalban 6y ago
Make a cogent and non-authoritarian case for even having a "decentralized content moderation" which doesn't pass the "is this an oxymoron?" smell test. > I'm building a large, chatroom-like service for TV. So the profit motive is likely the motivation for applying a "central" doctrine of acceptable discourse using a decentralized mechanism. > While I think that community moderation will inevitably lead to bubbles Which allows for e.g. an athiest community to have content that rips religion x's scripture to shred (and why not?) in the same planet that also has a religion-x community that has content that takes a bat to over-reaching rationalism. Oh the horror! Diversity of thought. "We simply can not permit this."

lawrencevillain 6y ago

Obviously upvoting and downvoting is not enough for adequate moderation. There's still the aspect of people trolling and generally posting horrible things online. There's a reason Facebook had to pay $52 million to content moderators for the trauma/ptsd they suffered.

neiman 6y ago

We (at Almonit) work on a self-governing publication system which would bring democratic control to content moderation. We just wrote about its philosophy earlier this week. https://almonit.com/blog/2021-01-08/self-governing_internet_organizations_part_I.html https://almonit.com/blog/2021-01-08/self-governing_internet_...

theylon 6y ago
This is an interesting concept. Are there any implementations of this?

deleted 6y ago

[deleted]

emaro 6y ago

Matrix published an interesting concept for decentralised content moderation [0]. I think this is the way to go. Edit: Discussed here [1] and here [2]. [0]: https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix-without-backdoors https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix... [1]: https://news.ycombinator.com/item?id=24826951 https://news.ycombinator.com/item?id=24826951 [2]: https://news.ycombinator.com/item?id=24836987 https://news.ycombinator.com/item?id=24836987

jrexilius 6y ago

Good piece. This line articulates the problem well: "without objectivity and consistency, moderation can easily degenerate into a situation where one group of people forces their opinions on everyone else, like them or not." And it gets to the core of the problem. Objectivity and consistency are extremely difficult to scale and maintain over time. They require constant reinforcement from environment, context, and culture.

zarkov99 6y ago
The central problem here is in what circumstances free people should give up the right to decide what information they can consume. This is not a question that can be answered easily but without first accepting it as the central issue we are not going to make meaningful progress.
- nindalf 6y ago
  That’s part of it. The other part is that free people say they don’t want to see illegal content (child exploitation imagery, terrorism, scams, sale of opioids etc). The platform needs to moderate to remove that. Then the same users also say they don’t want to see legal but distasteful (in their opinion) content like pornography, spam and so on. The platform then has to remove that as well. For most part platforms take decisions that will suit the majority of users.
  - Viliam1234 6y ago
    I don't want to see "legal but distasteful" content (and I would also add: annoying, boring, stupid, etc.)... and what I mean is that I don't want it to be shown to me... but I am okay if other people show it to each other. So instead of some global moderator (whether it be one person, or some complicated "democratic" process) deciding globally what is okay and what is not, I want many bubbles that can enforce their own rules, and the option to choose the bubbles I like. Then I will only be concerned about how to make the user interface as convenient as possible, so that the options are not only hypothetically there, but actually easy to use also by non-tech users.

EGreg 6y ago

Let's take one step back. Just like in the Title I vs Title II debate, let's go one step earlier. WHY do we have these issues in the first place? It's because our entire society is permeated with ideas about capitalism and competition being the best way to organize something, almost part of the moral fabric of the country. Someone "built it", now they ought to "own" the platform. Then they get all this responsibility to moderate, not moderate, or whatever. Compare with science, wikipedia, open source projects, etc. where things are peer reviewed before the wider public sees them, and there is collaboration instead of competition. People contribute to a growing snowball. There is no profit motive or market competition. There is no private ownership of ideas. There are no celebrities, no heroes. No one can tweet to 5 million people at 3 am. Somehow, this has mistakenly become a “freedom of speech” issue instead of an issue of capitalism and private ownership of the means of distribution. In this perverse sense, "freedom of speech" even means corporations should have a right to buy local news stations and tell news anchors the exact talking points to say, word for word, or replacing the human mouthpieces if they don't... Really this is just capitalism, where capital consists of audience/followers instead of money/dollars. Top down control by a corporation is normal in capitalism. You just see a landlord (Parler) crying about higher landlord ... ironically crying to the even higher landlord, the US government - to use force and “punish” Facebook. Going further, it means corporations (considered by some to have the same rights as people) using their infrastructure and distribution agreements to push messages and agendas crafted by a small group of people to millions. Celebrity culture is the result. Ashton Kutcher was the first to 1 million Twitter followers because kingmakers in the movie industry chose him earlier on to star in movies, and so on down the line. Many companies themselves employ social media managers to regularly moderate their own Facebook Pages and comments, deleting even off-topic comments. Why should they have an inalienable right to be on a platform? So inside their own website and page these private companies can moderate and choose not to partner with someone but private companies Facebook and Twitter should be prevented from making decisions about content on THEIR own platform. You want a platform that can’t kick you off? It’s called open source software, and decentralized networks. You know what they don’t have? Private ownership of the whole network. “But I built it so I get to own it” is the capitalist attitude that leads to exactly this situation. The only way we will get there is if people build it and then DON’T own the whole platform. Think about it!

jonathanstrange 6y ago

Since I'm using libp2p in Go for a side project, may I take this opportunity to ask how this could work in principle for a decentralized network? The way I see it, this seems to be impossible but maybe I'm missing something. For example, in my network, anyone can start a node and the user has full control over it. So how would you censor this node? The following ideas don't seem to work: 1. Voting or another social choice consensus mechanism. Problems: - Allows a colluding majority to mount DOS attacks against anyone. - Can easily be circumvented by changing host keys / creating a new identity. 2. The equivalent of a killfile: Users decide to blacklist a node, dropping all connections to it. Problems: - Easy to circumvent by creating new host keys / creating a new identity. 3. Karma system: This is just the same as voting / social choice aggregation and has the same problems. 4. IP banning by distributing the blocked IPs with the binaries in frequent updates. Problem: - Does not work well with dynamic IPs and VPNs. Basically, I can't see a way to prevent users from creating new identities / key pairs for themselves whenever the old one has been banned. Other than security by obscurity nonsense ("rootkit" on the user's machine, hidden keys embedded in binaries, etc.) or a centralized server as a gateway, how would you solve that problem?

JulianMorrison 6y ago
The way it works in Mastodon is that (1) not everyone runs a node, but there are many nodes, and they each have their own policies and can kick users off, and (2) nodes can blacklist other nodes they federate content from. This two level split allows node operators to think of most other users at the node level, which means dealing with far fewer entities. It provides users with a choice of hosts, but means that their choice has consequences.
yorwba 6y ago
> I can't see a way to prevent users from creating new identities / key pairs for themselves whenever the old one has been banned. You could prevent banned users from returning with a new identity by disallowing the creation of new identities. E.g. many Mastodon instances disable their signup pages and new users can only be added by the admins. If you don't want to put restrictions on new identities, you could still treat them as suspect by default. E.g. apply a kind of rate limiting where content created by new users is shown at most once per day and the limit rises slowly as the user's content is viewed more and more without requiring moderation. (This is a half-baked idea I had just now, so I'm sure there are many drawbacks. But it might be worth a shot.)

colllectorof 6y ago

It's pretty obvious that the tech crowd right now is so intoxicated by its own groupthink that these attempts to come up with "solutions" are going to have awful results. You don't even know what the problem really is. "I fear that many decentralised web projects are designed for censorship resistance not so much because they deliberately want to become hubs for neo-nazis, but rather out of a kind of naive utopian belief that more speech is always better. But I think we have learnt in the last decade that this is not the case." What you should have learned in the last decade is that social networks designed around virality, engagement and "influencing" are awful for the society in the long run. But somehow now the conversation has turned away from that and towards "better moderation". Engage your brain. Read Marshall McLuhan. The design of a medium is far more important than how it is moderated.

crmrc114 6y ago
"What you should have learned in the last decade is that social networks designed around virality, engagement and "influencing" are awful for the society in the long run." Yes, and don't forget the 24 hour news cycle with its focus on getting outrage and attention through fear. I did not know who Marshall McLuhan was until now- thanks for the tip!
- ardy42 6y ago
  > Yes, and don't forget the 24 hour news cycle with its focus on getting outrage and attention through fear. Yeah, social media is just one in a series of possibly misguided techno-social "innovations," and it probably won't be the last. My understanding is that groups like the Amish don't reject technology outright, but adopt it selectively based on its effects on their society (and will even roll back things they've adopted if they're not working out). Wider society probably would benefit from a dose of that kind of wisdom right now, after decades of decades of "because we can"-driven "innovation."
  - jl2718 6y ago
    They also force all 17-year-olds to go live with and as ‘the English’ for two years and then decide whether they want to go back, and what things should be brought back with them.

jancsika 6y ago

> Censorship resistance means that anybody can say anything, without suffering consequences. I can't even get to the heart of the poster's argument. That's because the shitty state of all current social media software defines "anybody" as: * a single user making statements in earnest * a contractor tacitly working on behalf of some company * an employee or contractor working on behalf of a nation state * a botnet controlled by a company or nation state It's so bad that you can witness the failure in realtime on, say, Reddit. I'm sure I'm not the only one who has skimmed comments and thought, "Gee, that's a surprisingly reaction from lots of respondents." Then go back even 30 minutes later and the overwhelming reaction is now the opposite, with many comments in the interim about new or suspicious accounts and lots of moderation of the initial astroturfing effort. Those of us who have some idea of the scope of the problem (hopefully) become skeptical enough to resist rabbit-holes. But if you have no idea of the scope (or even the problem itself), you can easily get caught in a vicious cycle of being fed a diet of propaganda that is perhaps 80% outright fake news. As long as the state of the art remains this shitty (and there are plenty of monetary incentives for it to remain this way), what's the point of smearing that mendacity across a federated system?

gfodor 6y ago

You can cut off a large part of abuse by just creating a financial based incentive. Pay to gain access, and access can be revoked at which point you need to repay (perhaps on a progressive scale - ie the more you are banned, the harder it is to get back in.) Your identity confers a reputation level that influences filters so what you post is seen more often, so there is value in your account and you don't want to lose it. The SA forums did this and it helped immensely with keeping out spam (though not a silver bullet.) Any system where any rando can post any random thing with no gates is going to be much more of a slog to moderate than one where there are several gates that imply the person is acting in good faith.

eternalban 6y ago

I fear that many decentralised web projects are designed for censorship resistance not so much because they deliberately want to become hubs for neo-nazis, but rather out of a kind of naive utopian belief that more speech is always better. But I think we have learnt in the last decade that this is not the case. If we want technologies to help build the type of society that we want to live in, then certain abusive types of behaviour must be restricted. Thus, content moderation is needed. Let's unpack this: Axiom: a kind of naive utopian belief [exists that asserts] that more speech is always better. But I think we have learnt in the last decade that this is not the case. False premise. The "naive belief", based on the empirical evidence of history, is that prioritizing the supression of speech to address social issues is the hallmark of authoritarian systems. Martin also claims "we have learned" something that he is simply asserting as fact. My lesson from the last 3 decades has been that it was a huge mistake to let media ownership be concentrated in the hands of a few. We used to have laws against this in the 90s. Axiom: By "we" as in "we want", Martin means the community of likeminded people, aka the dreaded "filter bubble" or "community value system". Who is this "we", Martin? Theorem: If we want technologies to help build the type of society that we want to live in, then certain abusive types of behaviour must be restricted. We already see that the "we" of Martin is a restricted subset of "we the Humanity". There are "we" communities that disagree with Martin's on issues ranging from: the fundamental necessity for freedom of thougth and conscience; the positive value of diversity of thoughts; the positive value of unorthodox ("radical") thought; the fundamental identity of the concept of "community" with "shared values"; etc. Q.E.D.: Thus, content moderation is needed. Give the man a PhD. -- So here is a parable of a man named Donald Knuth. This Donald, while a highly respected and productive contributing member of the 'Community of Computer Scientists of America' [ACM, etc.], also sadly entertains irrational beliefs that "we" "know" to be superstitious non-sense. The reason that this otherwise sane man entertains this nonsensical thoughts is because of the "filter bubble" of the community he was raised in. Of course, to this day, Donald Knuth has never tried to force his views in the ACM on other ACM members, many of whom are devout athiests. And should Donald Knuth ever try to preach his religion in ACM, we would expect respectful but firm "community filter bubble" action of ACM telling Mr. Knuth to keep his religious views for his religious community. But, "[i]f we want technologies to help build the type of society that we want to live in" -- and my fellow "we", do "we" not agree that there is no room for Donald Knuth's religious nonsense in "our type of society"? -- would it not be wise to ensure that the tragedy that befell the otherwise thoughtful and rational Donald Knuth could happen to other poor unsuspecting people who happen to be born and raised in some "fringe" community? "Thus, content moderation is needed."

iamsb 6y ago

My suggestion is to not indulge in any content moderation which is not illegal. Only take down content which is required by a court order. Limit use of automated content moderation only for easy to solve cases like child pornography. Why? It is fairly clear at this point that content moderation at internet scale is not possible. Why? A. Using other users to flag dangerous content is not working. Which users do you trust to bestow this power with? How do remove this power from them? How do you control it becoming a digital lynch mob? Can you have users across political, gender, other dimensions All mostly not solvable problems. B. Is it possible to use machine learning? To some extent. But any machine learning algorithm will have inherent bias, because test data will also be produced by biased individuals. Also people will eventually figure out how to get around those algorithms as well. The causality between content published on the internet and action in real world is not immediate. It is not like someone is sitting in a crowded place and shouting fire causing a stampede. As there is a sufficient delay between speech and action, we can say that the medium the speech is published in is not the primary cause of the action, even if there is link. Chances of direct linkage are fairly rare and police/law should be able to deal with those. Content moderation, at least the way Twitter has been trying to do, has not been effective, created lot of ways for mobs to enforce censorship, and there is absolutely no real word positive impact of this censorship is. Only use of this moderation and censorship has been for right to claim victimhood and gain more viewer/readership to be honest.

Pfhreak 6y ago
You realize that the approach you suggest pushes out a different set of people, right? For example, a soldier with PTSD may want an environment that moderates content. Or a journalist with epilepsy may want a platform where people don't spam her with gifs designed to trigger epilepsy when she says something critical of a game release.
- iamsb 6y ago
  I understand. Most of those can be achieved using privacy and sharing settings though and does not necessarily require content moderation.

JulianMorrison 6y ago

It's worth remembering that content moderation is an activity which causes people mental illness right now, because it's so unrelentingly awful. Attempts to decentralize this are going to be met with the problem that people don't want to be exposed to a stream of pedophilia, animal abuse, murderous racism and terrorist content and asked to score it for awfulness.

totemandtoken 6y ago
This point should be upvoted more. Decentralized moderation means decentralized pain. You can run standard philosophic thought experiments about "utility monsters" but at the end of the day a lot of online content causes real harm, not just to the direct victims but to the spectators. We'd need something like decentralized therapy in tandem with this type of moderation for it to even be considered remotely ethical, and even then I'm very skeptical.
okokok___ 6y ago
I agree, I think people underestimate the need to incentivize moderators. This is why I think some kind of cryptocurrency based solution to moderation is necessary.
- JulianMorrison 6y ago
  Moderators don't just need incentives. They need therapy and support and monitoring and breaks and ideally, pre-vetting for strong mental health and no pre-existing bees in their bonnet. I worry that just making it a way to earn bitcoins risks it becoming one more way for poor people to scrape together pennies at the cost of giving themselves PTSD.
  - okokok___ 6y ago
    good point. It's almost like mechanical turk but for moderation.

kstrauser 6y ago

I'm active with Mastodon and absolutely love its moderation model. In a nutshell: - It's made up of a bunch of independent servers, or "instances". The common analogy here is to email systems. - If you want to join the federation, stand up an instance and start using it. Voila! Now you're part of it. - My instance has a lot of users, and I don't want to run them off, so it's in my own interest to moderate my own instance in a way that my community likes. Allow too much in without doing anything? They leave. Tighten it so that it starts losing its value? They leave. There's a feedback mechanism that guides me to the middle road. - But my users can leave for greener pastures if they think I'm doing a bad job and think another instance is better. They're not stuck with me. The end result is that there are thousands of instances with widely varied moderation policies. There are some "safe spaces" where people who've been sexually assaulted hang out and that have zero tolerance for harassment or trolling. There are others that are very laissez faire. There's a marketplace of styles to choose from, and no one server has to try to be a perfect fit for everyone. I realize that this is not helpful information for someone who wants to run a single large service. I bring it up just to point out that there's more than one way to skin that cat. (That final idiom would probably get me banned on some servers. And that's great! More power to that community for being willing and able to set policies, even if I wouldn't agree with them.)

makeworld 6y ago
Couldn't have said it better myself. Mastodon appears to solve all the moderation problems I've seen raised about social media.

MrXOR 6y ago

The problem of censorship on Twitter and other social media can be solved with a good moderator like dang (thank you!)

mikeveilleux 6y ago

Having been part of the email ecosystem where well established spam filtering and reputation management systems have been in place for years, I've found it interesting how close these more recent conversations are to those we've had around email messaging abuse. > Thus, as soon as a censorship-resistant social network becomes sufficiently popular, I expect that it will be filled with messages from spammers, neo-nazis, and child pornographers (or any other type of content that you consider despicable). Unfortunately, I agree this is likely the case, and also agree with many of the other points where there's unlikely to be an agreed upon approach at scale. I feel the two most important aspects of any moderation are transparency and consistency. I'd always like to know what community I'm joining. We'll likely see more niche communities continue to pop up on centralized and decentralized networks where the moderation, content and community can be more tailored to their own expectations.