6 ms·

> How much do you understand about statistics and p-values? I suspect that it’s far far less than 38% of people who actually understand statistics to this leve

by maccard 22d ago

> How much do you understand about statistics and p-values? I suspect that it’s far far less than 38% of people who actually understand statistics to this level. I suspect if someone on HN went around and asked their co workers to explain what a P value is in 2 sentences, it would be less than 10% of a (presumably) highly educated workforce. I suspect about 40% of adults are unable to tell the difference between mean/median/ mode, or could answer the Monty hall problem, or even “if I flip a coin 3 times are the chances I get heads 3 times in a row”

applfanboysbgon 22d ago
I mean, the Monty Hall problem is a literal gotcha that trips up literal professors, that's a horrendous example to use as the baseline for "basic understanding of statistics". Other than that choice of example, I do agree in that I doubt anywhere near 40% of adults have basic statistical literacy. I've played in card game tournaments semi-professionally and just gambler's fallacy + results-oriented thinking alone make it so easy to take other people's money, and if you can't figure out such basic concepts as "getting tails once doesn't mean I'm due for a heads next flip" even when you're literally losing money, what are the chances of anyone else caring about understanding it when they're not even being given the hands-on reward-based reinforcement learning opportunity?
- maccard 22d ago
  Yeah I maybe should have ignored the Monty hall problem - although I’d guess if you’ve studied enough stats to know what a P value is, and how to measure it, you’ve come up against the Monty hall problem!
izacus 22d ago
Most HNers don't even understand what percentages are and throw out dumb statements like "mega corp can treat 1% of users like crap, it's a small number!"
ViscountPenguin 22d ago
In my experience, a not insignificant chunk of students actively in a stats class think that a p value is the probability of the null hypothesis. My money is on more like 3% of educated professionals.
- maccard 21d ago
  If you asked me for the definition of p value I could give it to you but if you asked me what it meant before this thread I would have said the same thing as you’ve said here.
jampekka 22d ago
10% may be a magnitude too high at least if we expect the explanation to be correct. For example in a sample of statisticians/epidemiologists only 12.5% got two basic features of p-values correct [1]. And there are a lot of studies showing similar levels of misunderstanding among people using p-values professionally. I use p-values daily. I can and do compute them using various methods, including by hand. But I'd still not be confident in my two sentence explanation. P-values are very unintuitive and very easy to get subtly wrong. [1] https://pubmed.ncbi.nlm.nih.gov/35991465/ https://pubmed.ncbi.nlm.nih.gov/35991465/
- maccard 21d ago
  P values are tricky, and I think you can get a lot out of knowing what they are without being able to compute them and use them reliably. But statisticians not knowing the basics of the Null Hypothesis is absolutely terrifying!

by maccard 22d ago

applfanboysbgon 22d ago

I mean, the Monty Hall problem is a literal gotcha that trips up literal professors, that's a horrendous example to use as the baseline for "basic understanding of statistics". Other than that choice of example, I do agree in that I doubt anywhere near 40% of adults have basic statistical literacy. I've played in card game tournaments semi-professionally and just gambler's fallacy + results-oriented thinking alone make it so easy to take other people's money, and if you can't figure out such basic concepts as "getting tails once doesn't mean I'm due for a heads next flip" even when you're literally losing money, what are the chances of anyone else caring about understanding it when they're not even being given the hands-on reward-based reinforcement learning opportunity?

maccard 22d ago
Yeah I maybe should have ignored the Monty hall problem - although I’d guess if you’ve studied enough stats to know what a P value is, and how to measure it, you’ve come up against the Monty hall problem!

izacus 22d ago

Most HNers don't even understand what percentages are and throw out dumb statements like "mega corp can treat 1% of users like crap, it's a small number!"

ViscountPenguin 22d ago

In my experience, a not insignificant chunk of students actively in a stats class think that a p value is the probability of the null hypothesis. My money is on more like 3% of educated professionals.

maccard 21d ago
If you asked me for the definition of p value I could give it to you but if you asked me what it meant before this thread I would have said the same thing as you’ve said here.

jampekka 22d ago

10% may be a magnitude too high at least if we expect the explanation to be correct. For example in a sample of statisticians/epidemiologists only 12.5% got two basic features of p-values correct [1]. And there are a lot of studies showing similar levels of misunderstanding among people using p-values professionally. I use p-values daily. I can and do compute them using various methods, including by hand. But I'd still not be confident in my two sentence explanation. P-values are very unintuitive and very easy to get subtly wrong. [1] https://pubmed.ncbi.nlm.nih.gov/35991465/ https://pubmed.ncbi.nlm.nih.gov/35991465/

maccard 21d ago
P values are tricky, and I think you can get a lot out of knowing what they are without being able to compute them and use them reliably. But statisticians not knowing the basics of the Null Hypothesis is absolutely terrifying!