AI glossary · Basic · Language models · Safety & alignment
Loop Transformer
A looped Transformer is a design that handles a problem by passing the input through the same block of Transformer layers (neural network layers) multiple times. It is also called 'recurrent depth.'
About this term
- Level
- Basic · Technical terms you often meet in AI news
- Category
- Models & generation
- Added
- 2026-10-15
What is different
In a standard Transformer, the input passes once through a line of distinct layers and then produces an answer. A looped Transformer passes the input repeatedly through the same block of layers. This lets it spend more computation on a single input without increasing model size. Because it reuses the same weights, the aim is for a model of the same size to 'think more deeply.'
Standard transformer vs. looped transformer
Standard transformerDifferent layers, once each
- Input passes through layers in a single line
- More compute requires a bigger model
Looped transformerSame layers, multiple times
- Reuses the same weights repeatedly
- Uses more compute while keeping model size
- Intermediate computation may not fully show up in the CoT
공통 Both are based on the transformer architecture
How it works
The input sentence is first split into tokens and turned into numeric vectors. After these vectors pass once through the shared block of layers, the result is fed back into the same block as input. This repeats a fixed number of times, or as many times as needed, before the final answer is produced. The intermediate computation during the loop remains only as a Hidden Representation inside the model (an internal state in numeric form), not as text.
Why it is getting attention
According to reports, OpenAI is said to be using this design in its next flagship model and is therefore putting effort into safety work. OpenAI has not officially confirmed using this technique. It is also a research topic: the GitSwarm paper used the Looped Transformer as one of its tasks, in which multiple agents run successive experiments to improve a neural network architecture. In short, it is drawing interest as a way to improve reasoning ability without making models larger.
Things to know (limitations)
The biggest concern is that monitoring becomes harder. A common approach today is to read the CoT (Chain-of-Thought), the written record of a model's reasoning, to catch abnormal behavior. In a looped transformer, not all of the internal repeated computation may show up in the CoT, which can weaken this kind of monitoring. According to reports, OpenAI limited the use of looped structures in earlier models to preserve human-readable reasoning records, and the technique also became a point of contention in a dispute over the firing of safety researchers. Keep in mind that much of what is currently known comes from media reports, not official announcements.
Example
On this site, the term came up in the news item 'OpenAI applies looped transformer to next-generation model… focuses on safety as reasoning becomes harder to monitor (report)'. According to the report, OpenAI is doing alignment work and strengthening the sandbox of its test environment before releasing an unreleased next-generation model, and a person familiar with the matter said about 10% of its compute has been allocated to alignment and safety work. The report also cited an employee's account that if the new model passes internal safety evaluations, it can be deployed after the U.S. government tests it for up to 30 days.