It's really the most fundamental rule for every type of machine learning task -- there should be both a (1) test set and (2) a separate holdout validation set.

It's really the most fundamental rule for every type of machine learning task -- there should be both a (1) test set and (2) a separate holdout validation set. No matter if the task is classification, sequence prediction, regression, manifold learning, clustering, whatever. Really surprising how so few amateur traders understand this fundamental principal. Scientists too.