AI glossary · Beginner · Language models
Parameter
Parameters are the internal numbers an AI model adjusts on its own during training. The knowledge and way of judging that the model has learned are stored in these numbers.
About this term
- Level
- Beginner · Core ideas for first-time readers
- Category
- Basics
- Fields
- Language models
- Added
- 2026-10-07
How it works
A model starts with parameters set to mostly random values. As it looks at training data, it calculates how far its output is from the correct answer and changes the parameters slightly in the direction that reduces that gap. Repeating this process a very large number of times builds up patterns such as language rules and knowledge in the parameters.
Why it matters
The "B" number (in billions) after a model's name usually refers to its parameter count. Models with more parameters tend to be able to learn more complex things, but they also need more memory and compute to run. That is why parameter count is often used as a measure for estimating a model's size and operating cost.
Things to know
A higher parameter count does not automatically mean a better model. Depending on the quality of the training data and the training method, a small model can sometimes beat a larger one. Also, some architectures, such as MoE, use only a selected part of their total parameters, so it is worth distinguishing between the total count and the number actually used.
Example
"Paradee," covered on this site, is an example of distilling an existing speech synthesis model into a small model with far fewer parameters. Reducing parameters this way makes it easier to run the model even on small devices such as phones.
Where it came up on this site
- briefing Mistral AI releases 'Mistral Large 4' public preview; open weights expected later this month 2026-10-07
- briefing AWS offers Z.ai's 753B MoE model 'GLM 5.3' on Amazon Bedrock 2026-10-07
- briefing Paradee: Distilling Kokoro-82M into an 8M-parameter single-voice TTS 2026-10-06