6 ms·
I've always found it a little simplistic that the default cut-off, in most statistical software, for whether something should be 0 or 1, is 0.5. (i.e. > 0.5 eq
by reactspa 6y ago
I've always found it a little simplistic that the default cut-off, in most statistical software, for whether something should be 0 or 1, is 0.5.
(i.e. > 0.5 equals 1, and < 0.5 equals 0).
This seems to be a "rarely-questioned assumption".
Is there a reason why this is considered reasonable? And is there a name for the cut-off (i.e., if I were to want to change the cut-off, what keyword should I search for inside the software's manual?)?
- curiousgal 6y agoFrom a stats perspective the cutoff is included in the coefficients. If you use a design matrix (add a column of 1s to your variables) you get in a non matrix notation (beta_01 + beta_1 X_1 +...) So the threshold can be considered beta_0. In the software, you can get classification models to output class probabilities instead of class labels. You can then use whatever threshold you like for to transform those probabilities to labels. You may see it refered to as "discrimination threshold". Varying that threshold is how ROC curves are constructed.
- cuchoi 6y agoThe threshold would be beta_0 on every case or only when you have subtracted the mean from your data?
- vhhn 6y agoYou don't want to demean your dependent (response) binary variable. So you almost always want to keep beta0 to control for any imbalance in your dependent var.
- cuchoi 6y agoI meant demeaning the independent variables. My understanding is that the beta_0 will have the meaning the curiousgal attach it only if you demean your independent variables.
- vhhn 6y agoI see. But I think after demeaning X, beta0 will just have a special meaning... log odds of the average case. Nothing more.
- deleted 6y ago[deleted]
- anthony_doan 6y agostatistical software never does this. Almost all statistical models give you probabilities and it is up to the domain to determine the cutoff. You can clearly see this in logistic regression it's giving you a probability not a 1 or 0. When a clinician gives me a dataset I build the model and leave it to them to do the cut off. It is not my domain or expertise to tell them where the cut off is. Plus it gives away the responsibility. I'm not responsible for it unless they don't know and are asking me for a reasonable cut off. An example is people opting for artificial in semenation because of problem. I believe the cut off is above 60% because of the trade off. You can read more about it in Regression Modeling Strategies by Dr. Frank Harrell.