7 ms·
A basic course in statistics will inform you of why a 99.99% accurate test should be looked at with skepticism when diagnosing a rare disease. Yet we see the fa
by voxl 2mo ago
A basic course in statistics will inform you of why a 99.99% accurate test should be looked at with skepticism when diagnosing a rare disease. Yet we see the fancy 9s and think somehow this many 9s is enough.
- ziofill 2mo agoSure, but that’s not what we are talking about
- ulrikrasmussen 2mo agoDiagnosis of rare diseases is a subset of "true" statements, no?
- cyanydeez 2mo agohe's referring to frequency. a 2.9T parameter processing a 1M token data will inevitably produce wrong "false" tokens, guaranteed. So even if you have a 99.99% pure date set of the "true" variety, the errors will aggregate to some very less than 99.99% characteristics. Realm of big numbers problem.
- elendilm 2mo agoReally? Laws of probability works against your argument. Such a basic course in statistics needs to be scrutinized.
- californical 2mo agoEvery breath you take, there is a 99.99% chance that everything is normal, and a 0.01% chance that you breathe mild acid which horribly burns and causes a massive coughing fit. It probably don’t cause long term damage unless that breath happened to be more important than normal, like while driving right as a child runs into the road. Would you act differently knowing that you had one of these occasional acid breaths? Even if it only happened once per day on average (0.001%)
- elendilm 2mo agoAnd your point is? Laws of probability doesn't discriminate. There is always a probability a planet killer GRB hits us and we all die. Should we take precautions in our everyday life. Or should we wear helmet while walking on the road pray a coconut not falling on our head is just 99.999%.
- ziofill 2mo agoYou are correct, but this is a made up example, not a practical matter. You’re falling into the same trap as the author.
- elendilm 2mo agoYou must have come to rescue him :) from the "incorrect" trap.
- maxbond 2mo agoIf 1 in 10,000 people have a disease, then a "test" which always reports the patient doesn't have the disease will be correct 99.99% of the time. "99.99% accuracy" should be "looked at with skepticism" in that it doesn't tell you what you need to know to understand the quality of a a test for a rare disease (a classifier under conditions of severe class imbalance); at a minimum, you would want to understand it's false positive and false negative rate, not (just) it's overall error rate. See example "A": https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy
- elendilm 2mo agoIf only stating the obvious was a fallacy :) You appear not to have understood probability theory my friend. You will never get 100% in this universe for anything. What if "its a simulation" or "a dream" arguments ensures you never acheive 100%. Bayes probability theory will be a good start for you. https://en.wikipedia.org/wiki/Bayes%27_theorem https://en.wikipedia.org/wiki/Bayes%27_theorem
- maxbond 2mo agoWell, I do agree that all measurements contain error, but the point wasn't that the error rate would be greater than 0% but that a single headline summary of error can't always distinguish between good and bad tests.