AI glossary · Basic · Language models
Chat template
A chat template is a set of rules that converts a conversation made of system, user and assistant messages into exactly the string format (including special tokens) the model saw during training.
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Infrastructure & using models
- Fields
- Language models
- Added
- 2026-10-12
How it works
During training, an Instruct model saw conversations in which special tokens marked 'the user's turn starts here, the assistant's turn starts here.' A chat template converts the list of messages we send into a long string in exactly that format. At the end, it adds a marker meaning 'now it's the assistant's turn' so the model starts writing a reply. The formats for marking tool calls or thinking are often defined in the template as well.
How messages reach the model
- 1Message listSystem, user, assistant
- 2Apply templateMark roles with special tokens
- 3Turn markerNow it's the assistant's turn to reply
- 4TokenizationTokenizer converts to IDs
- 5Model generationStops at the end-of-turn token
Where you see it
You can find it in the chat_template field of tokenizer_config.json in a Hugging Face repository, or in a separate template file; it is written in a template syntax called Jinja. In the transformers library, functions such as apply_chat_template apply it. GGUF files may store the template in their metadata, and you can also specify the template directly in the inference engine's settings. Cloud APIs handle this process on the server, so users only send the list of messages.
Things to know
If the template is wrong, the model still runs, but common symptoms are off-target answers or the model not stopping and going on to make up the user's lines too. Each model has a different template, so you should not reuse another model's format as is. Base models often have no chat template, or one that is meaningless. When fine-tuning, the same template must be used in training and serving to maintain performance.
Example
A developer loaded a new open-weight model into a local inference engine, and its answers did not end, continuing on into sentences starting with 'User:'. Checking the model card again, they found that this model uses its own dedicated chat template, but the engine was applying another model's template. Once they set the template from the repository's tokenizer_config.json in the engine settings, conversations returned to normal.