While I agree with your overall sentiment, it is important to understand the relationship between class (im)balance and ROC curve performance. A very short arti

While I agree with your overall sentiment, it is important to understand the relationship between class (im)balance and ROC curve performance. A very short article which does a great job explaining this is [0]. There are, of course, other performance metrics that are appropriate in the presence of class imbalance, such as precision-recall curves, so I wouldn't go so far as to say "metrics on biased data are going to be biased". Some metrics can correct for the class imbalance bias. Others can't. [0] https://www.researchgate.net/profile/Jake-Lever/publication/305716645_Points_of_Significance_Classification_evaluation/links/6011d0d8299bf1b33e2d3d0c/Points-of-Significance-Classification-evaluation.pdf https://www.researchgate.net/profile/Jake-Lever/publication/...