6 ms·

The author points out in the text that the NFL result is statistically significant and the NBA result is not. A larger random sample ought to fall closer to th

by ronaldx 12y ago

The author points out in the text that the NFL result is statistically significant and the NBA result is not. A larger random sample ought to fall closer to the line than a smaller sample: it would be statistically highly remarkable if the graphs did look similar with zeroed axes - the larger sample should look much flatter. Starting both graphs from zero is unhelpful if the goal is correct visual interpretation.

sesqu 12y ago
One could argue that the large difference in sample sizes (more than fivefold) justifies scale manipulation to bring the variances into line, but it doesn't appear to me that the author normalized on variance at all (the axis should be something like 800-2000).
- ronaldx 12y ago
  I agree with what you say. To correctly compare the variances, the scale should be adjusted according to the square root of the ratio of sample sizes. The author has not done this correctly, but I claim that zeroing the axis would be further from the correct graph.
  - sesqu 12y ago
    I generally feel that zeroing the axis is never wrong, but yes, in this instance a (properly) cut axis would be better than correct. However, it should be apparent at a glance, and the presence of the other two graphs complicates things as well, so I have some reservations.
  - craigmbooth 12y ago
    Ah, you are correct, thank you. Apparently I wasn't thinking clearly when I normalized the axes to have the same absolute ranges.
CognitiveLens 12y ago
With broad correlational data like this, it's important to know what significance threshold is being used, and what the significance value actually is. There are many possible analyses that can be run, and there will be some subset of those analyses that produce significant results by chance, which is why you need hypothesis-driven analysis and/or very conservative corrections for post-hoc analysis. Without the details, we can't evaluate significance claims.

by ronaldx 12y ago

sesqu 12y ago

One could argue that the large difference in sample sizes (more than fivefold) justifies scale manipulation to bring the variances into line, but it doesn't appear to me that the author normalized on variance at all (the axis should be something like 800-2000).

ronaldx 12y ago
I agree with what you say. To correctly compare the variances, the scale should be adjusted according to the square root of the ratio of sample sizes. The author has not done this correctly, but I claim that zeroing the axis would be further from the correct graph.
- sesqu 12y ago
  I generally feel that zeroing the axis is never wrong, but yes, in this instance a (properly) cut axis would be better than correct. However, it should be apparent at a glance, and the presence of the other two graphs complicates things as well, so I have some reservations.
- craigmbooth 12y ago
  Ah, you are correct, thank you. Apparently I wasn't thinking clearly when I normalized the axes to have the same absolute ranges.

CognitiveLens 12y ago

With broad correlational data like this, it's important to know what significance threshold is being used, and what the significance value actually is. There are many possible analyses that can be run, and there will be some subset of those analyses that produce significant results by chance, which is why you need hypothesis-driven analysis and/or very conservative corrections for post-hoc analysis. Without the details, we can't evaluate significance claims.