AI Trend Research

← AI glossary

AI glossary · Advanced · Language models · Efficiency & infra

Test-time compute

Test-time compute refers to using more computation at the stage when an AI model, after training, actually answers questions (inference time) in order to improve answer quality.

About this term
Level
Advanced · Specialist terms that need the inner workings
Added
2026-10-08
In plain words It is like working a problem several times on scratch paper and checking your work before answering on a test, instead of writing an answer as soon as you see the question. Spending more time helps with hard problems, but on easy problems it may just waste time.

How it works

Previously, improving performance mainly meant making models larger and training them longer. Test-time compute instead uses more computation at the moment an answer is produced. A typical method is to have the model go through a long reasoning process (CoT) before its final answer. Other methods include generating several answers and picking the best one, or having the model review and revise its own work.

Where is the extra compute spent?Scale up the model → Larger model; Larger model → Answer immediately; Question → Same model; Same model → Think longer; Think longer → Refined answer; Think longer → Same model(Review & retry)Compute spent at training timeCompute spent at answer timeScale up the modelMore trainingLarger modelAnswer immediatelyQuestionSame modelThink longerSolve several times &reviewRefined answerReview & retry
Where is the extra compute spent? The top row spends more computation at the training stage; the bottom row spends more computation at the moment the answer is produced.

Why it matters

Some point out that scaling up the model itself costs enormous amounts and yields diminishing returns. Test-time compute showed that even with the same model, giving it more time to think can improve performance on problems that require step-by-step thinking, such as math and coding. This idea is at the core of reasoning models, and "how long to let the model think" has become a new knob for adjusting performance.

Good to know

The more the model thinks, the slower the response and the more tokens it uses, raising costs. For simple questions, thinking longer brings almost no benefit, so it is important to adjust the amount of computation to the difficulty of the problem. Longer thinking also does not always get closer to the right answer; the model sometimes reasons at length in the wrong direction and gets it wrong.

Example

Many AI services offer separate modes such as "fast response" and "think deeply," or let users choose the reasoning effort. When you choose the deep-thinking mode, the model thinks for a long time before answering, and the extra computation used then is exactly test-time compute.

Related terms

Reasoning model CoT Inference Scaling laws RLVR Token