AI glossary · Basic · MLOps & LLMOps
Experiment tracking
Experiment tracking is the practice of automatically recording which data, code and settings were used each time an AI model is trained, and what the results were, so runs can later be compared and reproduced exactly.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Data, ops & platforms
- Fields
- MLOps & LLMOps
- Added
- 2026-10-08
How it works
Adding a few lines of logging code to a training script means information is saved automatically every time training runs once (this is called a 'run'). Typically it records hyperparameters (settings such as the learning rate), the code version, the data version, how the loss function value changes during training, the final evaluation metrics and the resulting model files. The records can be viewed as tables or charts in a web interface, which makes it easy to compare several runs side by side.
What to record in a single run
Input
- Code version
- Data version
- Hyperparameters (learning rate, etc.)
Results
- Change in loss function value
- Final evaluation metrics
- Resulting model files
Why it matters
Model development involves running dozens or hundreds of trials while changing settings little by little, so without records you quickly lose 'that setting that worked well yesterday.' Experiment tracking helps reproduce good results and find out which change improved performance. It also makes it easy for team members to share results, reducing the waste of running the same experiment twice. The model from the best run is registered in a model registry and moves on to the deployment stage.
Things to keep in mind
If you record only code and settings and leave out the data version, results often cannot be reproduced. Logging too much makes storage grow quickly, so it is good to decide in advance what to keep. In the LLM era, the same approach is used not only for training but also for comparing Evals scores across prompt changes.
Example
Tools such as MLflow and Weights & Biases are well-known experiment tracking tools. For example, when a researcher trains several times while changing fine-tuning settings, each run's loss curve and evaluation scores accumulate on a dashboard. The researcher then picks the best run, compares which settings differed, and uploads that model to the registry.