Cross validation

Purpose: training 100 or 1000s of models and evaluating them all on the test set states the risk of "overfitting the test set". To avoid that, we use cross-validation to assess an approximation of test error, without fitting the test dataset.

• Principles and Methodology of Machine Learning - David Gianazza (slides 238-246)
• Model selection - scikit-learn

K-fold

..

Leave-one-out

..