5 ms·

> But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph). Adjusting for age, Alzheimers dea

by RIMR 1mo ago

> But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph). Adjusting for age, Alzheimers deaths were 1/100 for taxi drivers, and 1/60 for the general public. That's extremely significant. However you feel about the graphs means nothing compared to the actual numbers presented here. > I'm sure it is a great study. Lots of science done. They do good graphs and all that. Your choice to be dismissively anti-science helps to explain the bad point you're making.

bluGill 1mo ago
> That's extremely significant. What makes you think that is significant at all? Anytime you take a lot of small subsets of a large random sample you expect to find outliers that look a lot like this. That is https://xkcd.com/882/ https://xkcd.com/882/ seems to better fix the data just as well.
- fc417fc802 1mo ago
  Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.
  - bluGill 1mo ago
    It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.
    - fc417fc802 1mo ago
      You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset. To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check. Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
      - bluGill 1mo ago
        The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.
  - roenxi 1mo ago
    It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value. In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01. [0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...
dekhn 1mo ago
Comparing 1/100 and 1/60 doesn't mean anything in terms of statistical significance (you would need to know more). They aren't being anti-science and it's low-quality to suggest they are.

by RIMR 1mo ago

bluGill 1mo ago

> That's extremely significant. What makes you think that is significant at all? Anytime you take a lot of small subsets of a large random sample you expect to find outliers that look a lot like this. That is https://xkcd.com/882/ https://xkcd.com/882/ seems to better fix the data just as well.

fc417fc802 1mo ago
Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.
- bluGill 1mo ago
  It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.
  - fc417fc802 1mo ago
    You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset. To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check. Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
    - bluGill 1mo ago
      The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.
- roenxi 1mo ago
  It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value. In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01. [0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...

dekhn 1mo ago

Comparing 1/100 and 1/60 doesn't mean anything in terms of statistical significance (you would need to know more). They aren't being anti-science and it's low-quality to suggest they are.