AI glossary · Basic · MLOps & LLMOps · Language models
Checkpoint
A checkpoint is a file that saves a model's weights (and training state) at a specific point during or after training. It is used to resume training, or to load and compare the model as it was at that point.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Infrastructure & using models
- Fields
- MLOps & LLMOps, Language models
- Added
- 2026-10-12
How it works
Training is a process of adjusting the weights little by little over many steps, so the state at that point is saved at regular intervals. A checkpoint for resuming training contains not only the weights but also information such as the optimizer state, learning rate schedule and current step count. So even if training stops due to a hardware failure, it resumes from the last checkpoint rather than starting over. After training ends, several checkpoints may be evaluated to pick the best one as the final model.
How checkpoints are used, in order
- 1Training progressAdjust weights at each step
- 2Periodic savingWeights, optimizer state
- 3Resume after interruptionFrom the last checkpoint
- 4Evaluate and selectPick the best point
- 5Export for deploymentLightweight, weights only
Where you'll see it
When you run training code, folders with step numbers such as checkpoint-1000 pile up. On Hugging Face, a released model itself is often called a 'checkpoint,' and intermediate training points are sometimes published as branches or revisions of the repository. In industry news, an 'internal checkpoint' often means a model version still in development that has not been released. In image generation communities, downloadable model files are also called checkpoints.
Things to know
Checkpoints for resuming training include the optimizer state and can be several times larger than files with weights alone, so storage needs planning. Serving only requires the weights, so a lightweight file is exported separately for deployment. The last checkpoint is not always the best; a point before overfitting starts may be better. Even models with the same name can give different results if their checkpoints (revisions) differ, so in production it is good practice to pin which revision you use.
Example
This site has covered a report that Google is testing an internal checkpoint of a next-generation model; here, a checkpoint refers to a model version at a specific point in development. In practice, teams running in-house fine-tuning save checkpoints at several steps, pick the one with the best evaluation score, export only the safetensors weights and deploy them for serving.
Where it came up on this site
- briefing Google testing 'Carbon,' an internal checkpoint succeeding 'Gemini 4 Argon'… "on par with Opus 5.5" (report) 2026-10-10
- briefing Qwen releases open-weight 'Qwen-Image-2.1-Turbo' for generating and editing images in 8 steps (community reports) 2026-10-09
- Papers Long-WAM: extending the context of World-Action models 2026-10-09