6 ms·

There are definitely some cases where it's important to use human-assessed metrics, like cooking a meal. But if you are the size of Facebook, Instagram, Tiktok

There are definitely some cases where it's important to use human-assessed metrics, like cooking a meal. But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like - you need something measurable that you can spread to multiple projects or even across the complete company culture. I fully agree with your point that metrics are typically not objective, like any data once it's used to communicate something. So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community? I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else?

jrochkind1 4y ago
> how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? My guess is that dang's moderation is a large part of it, and I think dang moderates (creates moderation policy and executes it) based on things that are "human-assessed" and not quantifiable.... > But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like You're probably right, but I think it's worth challenging this conventional wisdom. (Not necessarily here, we're not going to work it out, but still, I'll ask some rhetorical questions) Why is it too dangerous? Even if you have quantitative metrics, are you _sure_ this isn't still "letting individuals define what success looks like", since individuals decided how the metrics were defined and measured? So then, what's the difference? Why, specifically, is it too "dangerous" to have non-quantitative metrics defining what success looks like? Danger of... what? I would say facebook as it is, is actually incredibly dangerous to, like, human society. So... what kind of danger are we talking about? Danger to facebook's profits instead? Or what? > So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community? Sure. I don't know! I think that's the question the OP is meaning to ask too, if not completely answer. The OP suggests: > At the start of this, I said that people join your community for support, connection, opportunities to give back, and meaningful relationships. Those are the things they value, and those are the things you should value. And if you value them, you should measure them. > They’re not as easy to measure as engagement, sure, but they can be measured. The best part is that maximizing these metrics is always going to be good for your community. My point is that i'm not certain this should be short-circuited with "Well of course whatever metrics these are, they need to be quantitative and have the appareance of "objectivity".
bombcar 4y ago
> I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else? HN is successful because it can be moderated by one dang. Maybe a bonus helper can appear. But once you get so big that your dang can't do it all anymore, it will eventually fall apart.
marginalia_nu 4y ago
Why does it need to be measurable?
- kelseyfrog 4y ago
  HN in general isn't aware of operationalization[1]. It's an interesting quirk of computer science being a STEM field, but CS research is qualitatively different from physics, sociology, psychology, biology, zoology, and ag science. Rather, the typical CS student has a pre-theoretical conceptualization of quantification and data collection as it pertains to research methodology. Instead of relying on operationaluzation, the phenomena is considered strictly immeasurable. It leaves a lot on the table. In the examples above, for instance, there are a number of unsubstantiated assumptions as well as pros and cons that implicitly do not account for operationalization and it shows. 1. https://en.m.wikipedia.org/wiki/Operationalization https://en.m.wikipedia.org/wiki/Operationalization
  - jholman 4y ago
    Off-topic, but learning about classic examples of operationalization is when I realized that mostly psychology isn't a real science (yet). I believe the canonical example involved me learning the phrase "bobo doll".
    - kelseyfrog 4y ago
      How interesting, same with operationalization in physics too?
  - marginalia_nu 4y ago
    This still doesn't answer my question of why.

jrochkind1 4y ago

> how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? My guess is that dang's moderation is a large part of it, and I think dang moderates (creates moderation policy and executes it) based on things that are "human-assessed" and not quantifiable.... > But if you are the size of Facebook, Instagram, Tiktok, etc. it is too risky (and expensive) to let individuals define what success looks like You're probably right, but I think it's worth challenging this conventional wisdom. (Not necessarily here, we're not going to work it out, but still, I'll ask some rhetorical questions) Why is it too dangerous? Even if you have quantitative metrics, are you _sure_ this isn't still "letting individuals define what success looks like", since individuals decided how the metrics were defined and measured? So then, what's the difference? Why, specifically, is it too "dangerous" to have non-quantitative metrics defining what success looks like? Danger of... what? I would say facebook as it is, is actually incredibly dangerous to, like, human society. So... what kind of danger are we talking about? Danger to facebook's profits instead? Or what? > So the question should be what metric(s), if not engagement in itself, should we measure for to create a healthy online community? Sure. I don't know! I think that's the question the OP is meaning to ask too, if not completely answer. The OP suggests: > At the start of this, I said that people join your community for support, connection, opportunities to give back, and meaningful relationships. Those are the things they value, and those are the things you should value. And if you value them, you should measure them. > They’re not as easy to measure as engagement, sure, but they can be measured. The best part is that maximizing these metrics is always going to be good for your community. My point is that i'm not certain this should be short-circuited with "Well of course whatever metrics these are, they need to be quantitative and have the appareance of "objectivity".

bombcar 4y ago

> I wonder if it's less about metrics and more about principles; how does HN keep the level of quality so high? Is it because we share an interest? Because of the quality control? The UI/UX of the site? Something else? HN is successful because it can be moderated by one dang. Maybe a bonus helper can appear. But once you get so big that your dang can't do it all anymore, it will eventually fall apart.

marginalia_nu 4y ago

Why does it need to be measurable?

kelseyfrog 4y ago
HN in general isn't aware of operationalization[1]. It's an interesting quirk of computer science being a STEM field, but CS research is qualitatively different from physics, sociology, psychology, biology, zoology, and ag science. Rather, the typical CS student has a pre-theoretical conceptualization of quantification and data collection as it pertains to research methodology. Instead of relying on operationaluzation, the phenomena is considered strictly immeasurable. It leaves a lot on the table. In the examples above, for instance, there are a number of unsubstantiated assumptions as well as pros and cons that implicitly do not account for operationalization and it shows. 1. https://en.m.wikipedia.org/wiki/Operationalization https://en.m.wikipedia.org/wiki/Operationalization
- jholman 4y ago
  Off-topic, but learning about classic examples of operationalization is when I realized that mostly psychology isn't a real science (yet). I believe the canonical example involved me learning the phrase "bobo doll".
  - kelseyfrog 4y ago
    How interesting, same with operationalization in physics too?
- marginalia_nu 4y ago
  This still doesn't answer my question of why.