And even the most fancy statistical methods are generally about proving that some mild generalization of linear regression is sufficient to solve a particular p

And even the most fancy statistical methods are generally about proving that some mild generalization of linear regression is sufficient to solve a particular problem. Non-linear models often result in a great increases in model complexity which leads to issues of overfitting, computational intractability, and instability.