Decision models: AI that answers from a set of options instead of writing text

Why it was needed: "I only need it to pick one answer"

Much software does not want long text from AI. It just needs a judgment that picks one of a fixed set of options: which department to route this inquiry to, whether this comment is hate speech, which emotion category this sentence belongs to. But if you use a generative LLM (large language model) for this kind of task, the model writes its answer by producing tokens (chunks of text) one at a time, and the program then has to parse that text again. This is slow and expensive, and there is also a risk the model answers in an unexpected format.

The degree of confidence is another problem. To filter classification results by a threshold, you need to know "how certain is it" as a number. When you ask a generative model for its confidence, it answers in words (verbalized confidence), and there is no guarantee this value matches the actual accuracy.

The decision model is a model type aimed at this gap. A study on social science text annotation paper introduces it as "a new model type built for categorical question answering," explaining that instead of free text it returns an option, a probability distribution over the entire label set, and a confidence score, at a cost that is a tiny fraction of frontier models. Note that the term "decision model" is also used with entirely different meanings in legal informatics and risk analysis. This note covers only the decision model as an AI model type.

Key idea: a multiple-choice answer sheet instead of an essay exam

If a generative LLM is a test-taker writing essay answers, a decision model is a test-taker who fills in "the probability this is the answer" in each box of a multiple-choice answer sheet. The examiner (the developer) sets the question and the options, and the model only scores each option. The program grading the answer sheet does not need to read any text; it can pick the highest probability or hand the case to a person if the probability falls below a threshold.

Related studies call these "typed" decisions. The form of the answer is declared in advance for each question: yes/no, one of several options, an ordered grade, and so on. Jev (TypeSafe AI's System One decision model) first introduced this interface for routing, content moderation and classification (triage), and open implementations followed paper.

The name "System One" comes from "System 1," which refers to fast, intuitive thinking. It means an approach that takes one look and answers right away. The SanSi study paper also describes a single forward pass as "fast, intuitive System 1 thinking."

How it works: read once and score each option

First, the developer declares the question and options. Each option usually comes with a short label and a definition stating when that label should be chosen paper. For example, the label might be "Urgent" and the definition "an outage report requiring action within 24 hours."

Second, the model reads the input text, the question and the options all at once. A generative model goes through a "decoding" stage in which it writes the answer one token at a time, but a decision model only goes through the "prefill" stage that processes the input. A study on prefill-only models paper explains that because such models score every candidate on the menu in a single forward pass and do no decoding, the cost of one call is 10–100 times lower than that of a generative model of the same size.

Third, a "decision head" attached to the end of the model outputs a probability for each option paper. The program takes this probability distribution as is and can pick the highest answer, or build a cascade that hands the case to a stronger model or a person when confidence is low.

How it developed: evaluation studies from fall 2026

Once Jev appeared, evaluation studies examining "is it actually usable?" followed in quick succession. A computational social science study paper compared the first commercial decision model and two open-weight models (models whose weights are publicly released) with 19 LLMs on 18 classification tasks (7,977 items). The decision models trailed the best LLM for each task on 14 of 15 evaluation tasks, with a median gap of 11.6 points in macro-F1 (a score that averages accuracy evenly across categories). In exchange, measurement cost was 44 times lower at the median. Their confidence was better calibrated than the verbalized confidence of 16 of the 19 LLMs. Being well calibrated means that answers given with "90% confidence" are actually correct close to 90% of the time. However, on a task judging empathy in peer counseling conversations, they performed poorly while showing high confidence.

The hate speech moderation study HATEDECIDE paper had somewhat more favorable results. Of four datasets, there was only one where a commercial LLM statistically significantly outperformed all decision models, and on diagnostic tests the best hosted decision model came within 1.6 macro-F1 points of the best commercial LLM, with about 97% lower inference cost. However, supplying a definition of hate speech changed up to 28% of predictions without consistently improving performance, and splitting the question into several questions produced significant gains in only 20% of comparisons.

The broadest evaluation is JEVal paper. It evaluated 25 model configurations on 11,257 cases drawn from 36 datasets across 10 application areas. Decision models were most competitive on tasks that can be judged from the given evidence alone, and weak on tasks requiring expert knowledge or honest uncertainty estimation. They tended to get the most plausible answer right while heavily inflating its probability. The same study also proposed InnerJev-4B and InnerJev-27B, built on open-weight models.

Around the same time, attempts to change the architecture also appeared. SanSi paper uses "looping," applying the same layers repeatedly, so the model can revise its internal state before settling on an answer without generating tokens. The researchers call this "System 1.5 thinking." On 10,027 decisions from 59 sources it achieved 72.0% accuracy, 13.5 points higher than a non-looping model with the same architecture, and 1.8 points behind a model with three times as many parameters (the values that indicate a model's size).

Where things are heading

First, "how the options are written" has become a research topic. The "Labels beat definitions" study paper showed that open decision models answer according to label names rather than option definitions (option-label bias). Deleting all the definitions left accuracy nearly unchanged (0.8559 vs. 0.8487), and renaming the options to A and B actually raised accuracy by 0.1511. The only difference between two codebases was whether options were written as "{label}: {definition}" or as the definition alone, and changing just this wording removed the bias without touching the weights. This means a small difference in prompt format changes the decision rule itself.

Second is research that aims to decide where to spend compute before deployment. The prefill-only model study paper showed that when the input text stays the same and only the candidate list (menu) changes, simply renormalizing the probabilities from the first computation to the new menu predicts the changed accuracy to within 4.2 points. This property lay in the rankings rather than in the probability values themselves, and generative models of the same size did not have it. At the same cost, a confidence-based cascade was better than asking the same model again, and refining the menu was better than scaling up the model.

Third is the question "is a separate model even necessary?" LLM-as-Jev paper showed that calibrated decisions can be extracted simply by reading a general LLM's next-token probabilities over numbers in parentheses. An untrained 4B model performed similarly to a community Jev-style model built on the same backbone, and it also handled multimodal decisions involving images. Fine-tuning had a large effect only in specific situations, such as weak models or intent classification with a very large number of options.

Limitations and open issues

The biggest issue is how much the confidence can be trusted. Outputting confidence as a number is a strength of decision models, but JEVal paper reported a tendency to inflate probabilities, and the social science study paper reported tasks where poor performance was hidden beneath high confidence. Any system that filters by a confidence threshold needs to validate this separately for each task.

Second is error accumulation in systems where small judgments pile up. JEVal found that in the multi-turn τ-bench environment, fast judgments reduced median episode time, but task success rates fell as judgment errors accumulated along long trajectories. In a large-scale social simulation, individual response prediction was close to strong LLMs, but the models were weak at user profiling and had larger aggregate estimation errors and systematic biases. This is a point to watch especially when using them as components of AI agents.

Third, the interface behaves counterintuitively. You might expect that writing definitions would make the model follow those rules, but in practice models followed the labels paper, or definitions only shifted predictions without improving performance paper. Developers need to verify the assumption that "well-written definitions are enough" through testing.

Further study

To get the big picture first, it is best to start with JEVal paper. It summarizes, across many domains, which decisions these models are strong at and where they break down. Then, for real-world applications, read the computational social science annotation study paper together with the hate speech moderation study HATEDECIDE paper to see the trade-offs among accuracy, cost and confidence in concrete numbers.

For developers, two papers are directly relevant to practice: the "Labels beat definitions" study paper, which covers the pitfalls in how option labels and definitions are written, and the prefill-only model study paper, which covers candidate menu design and compute allocation. Finally, if you are interested in architecture, compare SanSi paper, which extends thinking time through repeated computation, with LLM-as-Jev paper, which uses a general LLM as is like a decision model; this lets you judge for yourself the question of whether to use a dedicated model or a general-purpose LLM.