← Deep dives

Deep dives · 2026-11-15 · Advanced · By bombong

No-CoT: what you gain and lose when reasoning is not written out

No-CoT refers to not writing out the reasoning process. It covers training on data without worked solutions, moving reasoning into the model's internal representations, and answering directly in a single computation. It reduces token cost and latency, but removes the reasoning record that people can read and monitor. This note lays out that trade-off.

Contents
Key points
  • No-CoT is not the name of a single technique. It is an umbrella term for the choice of "not writing out reasoning" in three places: data, output and decision models.
  • When using no-CoT data without worked solutions, a single Mixed SFT stage that combines no-CoT and long-CoT data reached a higher performance ceiling after RLVR than the more complex next-chunk reasoning RL, while using more than 60 times less training compute.
  • Implicit CoT saves tokens, but training can collapse as latent tokens become similar to one another. SIM-CoT reduced this problem by adding step-level supervision only during training.
  • Removing CoT also removes CoT monitoring as a safety mechanism. Conversely, even when CoT is present, there is no guarantee that its content faithfully reflects the actual decision process.
  • In practice, "how much to have the model write" should be decided based on task difficulty, the need for auditing and model size. Strategies should not be chosen based only on intermediate training results.

Why No-CoT deserves separate attention

CoT (Chain-of-Thought) is an approach in which the model writes out intermediate steps before giving an answer. As it became known that reasoning models solve hard problems with long CoT, the intuition that "having the model write longer thoughts makes it better" spread. But CoT is not free. Every token written adds compute cost and latency, and training requires data with step-by-step solutions.

No-CoT looks at this cost from a different angle. Most of the world's data has no CoT annotations. Textbook derivations and model solutions are rich in reasoning, but they are not organized as a "thought process" format the model can imitate. A study revisiting training strategies for no-CoT data paper calls such material "no-CoT data" and addresses separately how to use it in training.

The same question arises for outputs. Where fast decisions are needed, such as routing or classification, having the model write long solutions can be wasteful. So research is also exploring moving reasoning into hidden representations (the model's internal representations), or producing the answer in a single computation altogether. This note groups these three threads under one question, "No-CoT": what do you gain and what do you lose when reasoning is not written out?

Core idea: three places where thinking is not written out

Let's start with an analogy. In a math exam, one student writes out every step, another calculates in their head and writes only the answer, and a third picks the answer as soon as they recognize the problem type. All three "think." The difference is whether that thinking is left on paper, and if so, how much. No-CoT is the choice to move from the first student toward the last.

This choice happens in three places. The first is data: the question is how to train on no-CoT data that has no worked solutions. The second is output: implicit CoT, where the model continues its reasoning with latent tokens (internal vectors rather than words people can read) instead of text, belongs here. The third is decision models: there are "System 1" models that, instead of generating text one token at a time, return a probability distribution over predefined options in a single forward pass. The Jev-style decision model audit study paper covers this type.

The key point is that all three places involve the same trade-off. Written reasoning costs more but can be read and checked by people. Unwritten reasoning is fast and cheap but cannot be read. In between, there are also approaches that strip only the unnecessary steps from long CoT. So it is more accurate to see No-CoT not as an either-or question of "use CoT or not," but as a question of which point to choose along a single axis.

How much reasoning is written out

← Writes long reasoningWrites no reasoning →
1234
  1. Long CoT
  2. Pruned CoT
  3. Implicit CoT
  4. System 1 direct answer
Further left, more reasoning is written out, making it easier to read and monitor; further right, it is faster and cheaper. Dot positions are relative examples.

How it works

Start with the data side. Among the methods proposed for using no-CoT data is next-chunk reasoning RL. The model generates its own hidden reasoning before the next chunk of text, and is rewarded based on how well that reasoning helps predict the next chunk. It is an attempt to draw out "thinking" from text without worked solutions using reinforcement learning. However, the study on training strategies for no-CoT data paper compared it under controlled conditions with a simpler alternative, Mixed SFT. Mixed SFT trains on no-CoT data and long-CoT data together in a single SFT (supervised fine-tuning) stage. The result: Mixed SFT had a higher performance ceiling after RLVR (reinforcement learning with verifiable rewards), and required more than 60 times less training compute. This advantage held not only for math reasoning but also for out-of-domain reasoning tasks.

On the output side, implicit CoT is the main example. Because reasoning steps are carried forward as latent tokens rather than written text, it saves tokens. But its performance has lagged behind explicit CoT, so it has not been widely adopted. SIM-CoT paper diagnosed the cause as follows: as the number of latent tokens increases, the latent representations become similar to one another and lose semantic diversity, and training collapses. The fix is step-level supervision that attaches an auxiliary decoder only during training to align each latent token with its corresponding explicit reasoning step. The auxiliary decoder is removed at inference, so there is no extra cost. This method improved Coconut's performance by 8.2% on GPT-2 and CODI's by 3.0% on LLaMA-3.1 8B. On GPT-2, it was 2.1% higher than the explicit CoT baseline with 2.3 times better token efficiency.

Decision models sit at the far end. According to the Jev-style model audit study paper, these "System 1" models return, in a single forward pass rather than through autoregressive generation, a probability distribution over a predeclared schema (options, scores and so on). The study compared them with generative "System 2" baselines. The baselines included scoring by the log probability of a single token and producing structured output via JSON or CoT.

How the field developed: from writing less to not writing at all

Across the studies in the input, the thread runs from "how can we shorten long written reasoning?" to "how do we handle reasoning that was never written out?"

The most modest starting point is the study on efficient long CoT for small language models paper. It addressed problems that arise when distilling the long CoT of reasoning models such as DeepSeek-R1 into SLMs (small language models). Long CoT contains many redundant steps, such as overthinking, which makes it hard for low-capacity models to learn. So the study pruned unnecessary steps and had the SLM itself select useful training data in an on-policy way. As a result, redundant reasoning dropped sharply while performance remained competitive. This is the stage of making models write less CoT, not removing it.

SIM-CoT paper went a step further and fixed the instability of implicit CoT, which moves reasoning out of text. Around the same time, a study on CoT monitorability paper asked the question from the other side. It systematically examined whether models faithfully put into words in their CoT the factors they actually used in their decisions, and whether CoT-based monitors reliably catch bad behavior. It showed that the premise that written reasoning is useful for safety may not hold.

The 2026 study on no-CoT data paper shifted the focus to data. It showed that using material without worked solutions does not necessarily require sophisticated reinforcement learning, and that high accuracy before RLVR does not mean high accuracy after RLVR. No-CoT training strategies should therefore be evaluated on results after the full post-training pipeline.

Notable numbers from No-CoT research

Over 60xTraining compute saved by Mixed SFT
2.3xSIM-CoT token efficiency (GPT-2)
+8.2%Coconut improvement with SIM-CoT
These are research results cited in the text. Each number comes from the experimental conditions of the respective paper.

Where things are heading

Recent research is moving in three broad directions. The first directly measures whether written reasoning is actually useful. A study on post-training and verbalized computation paper compared the SFT, DPO and RLVR checkpoints of Olmo-3-7B-Instruct in turn, looking at multi-digit addition. Carry values could be read out from internal activations with similar accuracy for both SFT and RLVR. But manipulating that information did not consistently change the output. In contrast, CoT produced by RLVR yielded higher accuracy than CoT produced by SFT or DPO, even when handed to other checkpoints. This means that the information being present internally and the written reasoning being useful are separate matters.

The second direction verifies decision models that do no generation at all. In the Jev-style model audit study paper, System 1 models with 4 billion or more parameters were faster and better calibrated under syntactic variation (ECE 0.030–0.087). The log-probability baseline scored 0.144 and the JSON baseline 0.378. However, the confidence these models report was not a value reflecting actual accuracy, but the spread of the probability distribution converted by a fixed formula. Also, when several questions were bundled into one request, the 0.8B model flipped 11.7–22.4% of its binary decisions. For the 4B model, this rate was 0.1–0.2%.

The third is a governance-side effort to preserve CoT. A proposal for CoT monitoring transparency disclosures paper argues that CoT monitoring is the most valuable tool for overseeing frontier AI but is under threat. It therefore proposed five metrics for developers to report. The metrics cover external access to CoT and specification of the underlying architecture, the naturalness and faithfulness of CoT, and the detection power of monitoring and its resulting effects. A review of the current disclosures of Anthropic, OpenAI and Google DeepMind found that none covered all five metrics. However, each metric was reported by at least one of them. The tension on the other side of No-CoT research is that these metrics may lose meaning as reasoning moves into latent space.

Limitations and open issues: when to use it and when to avoid it

The biggest issue is monitorability. If a model does not write out its reasoning, you lose the option of reading the CoT to catch bad behavior such as shortcuts or sycophancy. That does not mean writing a CoT makes a model safe. As the CoT monitorability study paper points out, models do not always faithfully put the factors behind their decisions into words. Monitors can also be too sensitive or not sensitive enough, and they can be fooled by long, elaborate reasoning. So what No-CoT gives up is not "perfect transparency" but "an imperfect yet usable window for monitoring."

Performance also comes with conditions. Without step-level supervision, implicit CoT can collapse as its latent tokens become too similar to one another paper. When small System 1 models answer several questions in one batch, the questions interfere more with each other, and the confidence a model reports does not match its actual accuracy paper. So using confidence values directly as thresholds is risky. When training on no-CoT data, high accuracy before RLVR does not mean accuracy will also be high after RLVR. Choosing a strategy based on intermediate scores is a common mistake paper.

As practical guidance: for routing and classification, where a model must decide quickly among fixed options, answering directly is the right fit. But check the model size and how questions are batched first. For work that needs multi-step calculation or an audit of the reasons behind decisions, keeping an explicit CoT is the better choice. If cost is a concern, first consider cutting redundant steps paper rather than removing CoT altogether. And if the training data lacks worked solutions, it is more cost-effective to start with a simple baseline such as mixed SFT than to jump straight into designing complex RL.

Checklist before adopting No-CoT

Task

  • Is it a quick decision among fixed options?
  • Does it require multi-step calculation?
  • Do the reasons behind decisions need to be audited?

Model and training

  • Have you avoided batching several questions into a small model?
  • Are you treating confidence values as actual accuracy?
  • Did you compare strategies using results after RLVR?

Alternatives

  • Have you tried cutting redundant steps before removing CoT?
  • Have you run a simple baseline such as Mixed SFT?
Items to check before reducing or removing reasoning.

Further reading

To understand the data side of No-CoT, start with the study on training strategies for no-CoT data paper. It covers, in a single paper, the definition of "no-CoT data," the idea behind next-chunk reasoning RL, and a controlled comparison with Mixed SFT. The lesson that you must evaluate after the full post-training pipeline also comes from this paper.

On the output side, SIM-CoT paper explains why implicit CoT was unstable and how supervision used only during training fixes it. For the earlier step of "writing less," the study on efficient long CoT for small language models paper is short and easy to read.

To see the other side of the trade-off, read the CoT monitorability study paper and the transparency disclosure proposal paper together. To see how the usefulness of internal information differs from that of written reasoning, read the study on post-training and verbalized computation paper. For the actual properties of decision models that do not generate text, read the Jev-style model audit study paper.

Related terms

CoT Reasoning model RLVR SFT Distillation Interpretability Test-time compute Decision model

References

Foundational papers

Recent research

Written by AI from the listed references and reviewed. Please check the original sources for details.