AI Trend Research

← AI glossary

AI glossary · Advanced · Language models · Generative media

Autoregressive model

An autoregressive model produces its output by repeatedly looking at what it has generated so far, predicting the next piece, appending that piece to the input, and predicting the next one again. Most of today's LLMs write text this way.

About this term
Level
Advanced · Specialist terms that need the inner workings
Added
2026-10-08
In plain words It is similar to a word-chain game, where each player looks at the previous word and adds the next one. But while a word-chain game looks only at the word just before, an autoregressive model looks at everything from the beginning up to now to choose the next piece.

How it works

When a user enters a prompt, the model looks at the whole text, calculates the probability of what is likely to come as the 'next token', and picks one. It appends the chosen token to the end of the existing text, feeds the longer text back into the model, and picks the next token again. This repeats until a signal meaning 'end' appears or a preset length is reached. The name 'autoregressive' comes from the fact that the model uses the output it just produced as its own input. Re-reading all earlier content every time would be very wasteful, so techniques such as the KV cache, which stores and reuses information already computed for earlier parts, are used alongside it.

A loop that writes one token at a timePrompt → Text so far; Text so far → LLM; LLM → One next token; One next token → Text so far(Append); One next token → Completed answer(End signal); KV cache → LLMPromptInitial inputText so farLLMCompute next-tokenprobabilitiesOne next tokenCompleted answerKV cacheReuse earliercomputationAppendEnd signal
A loop that writes one token at a time The model looks at the text so far, picks one next token, appends that token to the text and feeds it back into the model. When the end signal appears, the loop stops and the result is returned.

Why it matters

With nothing more than the simple goal of 'predicting the next token', a model can learn from the vast amount of text on the internet, which makes large-scale pre-training easy because no separate labels are needed. Most Transformer-based LLMs follow this approach, and many models also convert images, audio and video, not just text, into tokens and generate them the same way. For this reason, the autoregressive approach can be called the most basic generation principle of today's generative AI.

Things to know

Because tokens must be produced one at a time in order, longer outputs take more time and are hard to parallelize. To reduce this, techniques such as speculative decoding are used, where a small model guesses several tokens ahead and a large model checks them all at once. Also, a token chosen wrongly early on cannot be undone and later tokens keep building on it, so errors can accumulate. Because of these limits, there is also active research on applying the diffusion model approach, which refines the whole output simultaneously over several steps, to language generation.

Example

When you ask a chatbot a question, you can see the answer flow onto the screen one character or one word at a time. This is because the model generates tokens one by one in an autoregressive way and shows each right away. Among the papers featured on this site, 'In-Distribution Forcing' deals with how to manage the KV cache when generating long videos at test time, showing that video can also be generated autoregressively by looking at earlier parts and continuing with the next. In contrast, research on diffusion-based language models such as 'ALoDLM' belongs to the effort to find alternatives to the autoregressive approach.

Where it came up on this site

Related terms

LLM Token Transformer KV cache Speculative decoding Diffusion model