AI glossary · Beginner · Classic ML, recsys & search
Cross-validation
Cross-validation is a way to measure performance more reliably by splitting data into several parts and training and evaluating the model several times, taking turns using each part for evaluation.
About this term
- Level
- Beginner · Core ideas for first-time readers
- Category
- Basics
- Added
- 2026-10-09
How it works
The most common method is K-fold cross-validation. For example, the data is split into 5 parts, one is held out for evaluation, and the model is trained on the other 4. Repeating this 5 times while changing the evaluation part means every piece of data is used for evaluation once. Finally, the 5 scores are averaged, and how much the scores vary is also checked.
5-fold cross-validation steps
- 1Split into 5 foldsDivide the data evenly
- 21 fold for evaluationTrain on the other 4 folds
- 3Record the scoreMeasure performance on the evaluation fold
- 4Rotate folds, repeat 5 timesEvery fold is evaluated once
- 5Check mean and spreadDecide using all 5 scores
Why it matters
If you split the data only once for evaluation, the score can vary greatly depending on which data ended up in the evaluation set. Cross-validation combines the results of several splits, so it is less subject to luck. It is especially useful when data is scarce, because all the data can be used for both training and evaluation. It is also used to filter out overfitted models when choosing model settings (hyperparameters) or comparing several models.
Good to know
Because the model has to be trained several times, time and compute costs rise accordingly. For that reason it is used mainly with relatively small models or tabular data rather than when training a huge LLM from scratch. Data with a time order must be split in order so that future data is never used to evaluate the past, and when there is class imbalance, a stratified split is used to keep the class ratios the same in each fold. Be careful: if preprocessing is done on the full dataset before splitting it into folds, data leakage can occur.
Example
Suppose a hospital is building a disease-risk prediction model from records of a few hundred patients. With so little data, a single split would make the score depend heavily on chance, so the team uses 5-fold cross-validation to look at both the average performance and its spread, and uses the results to decide between gradient boosting and a decision tree. Folds are split by patient so that records from the same patient never end up in both the training and evaluation sets.