AI glossary · Beginner · Language models · Efficiency & infra
SLM
An SLM (small language model) is a language model with relatively few parameters, so it is lightweight and runs fast. There is no fixed size threshold; "small" is a relative term compared with large LLMs.
About this term
- Level
- Beginner · Core ideas for first-time readers
- Category
- Basics
- Added
- 2026-10-08
How it works
Most use the same Transformer-based architecture as LLMs, just smaller. To get useful performance at a small size, high-quality training data or synthetic data is chosen carefully. Distillation, in which a small model (the Student) learns by imitating the answers of a large model (the Teacher), is also often used. Shrinking the size further with quantization lets them run even on laptops or smartphones.
Why it matters
A smaller model is cheaper to run and answers faster. It can run inside a device without an internet connection, making it suitable for on-device AI that does not send personal information to external servers. For tasks with a defined scope, such as document classification or summarization, an SLM is often fine-tuned and used nearly as well as a large model. That is why many companies release a separate lineup of small models alongside their large ones.
Things to keep in mind
Because it can hold less knowledge, it tends to be weak at broad general knowledge or complex multi-step reasoning. Hallucinations, where it makes up things it does not know, can also be more frequent than in large models. Since the threshold for "small" differs by company, it is hard to guess performance or required hardware from the name alone. It is best to test it directly on the tasks you will actually use it for before choosing.
Example
SLMs are widely used for features that must be processed right on the device, such as message summaries or reply suggestions on smartphones. Companies also deploy SLMs on their own servers to avoid sending internal documents outside, fine-tune them on work documents, and use them to assist with classification and search.