Hallucination: Why It Happens and How to Reduce It

Why it matters: models that are plausibly wrong

Hallucination is when a large language model (LLM) produces fluent output that is factually wrong. Because the grammar and tone are fine, readers find the errors hard to notice. This makes it a matter of trust, not just performance. 'Why Language Models Hallucinate paper' points out that hallucination persists even in the latest systems and undermines trust.

It is commonly expected that scaling up models and data will reduce hallucination on its own. However, the neural diversity study paper starts from the observation that language models keep hallucinating even as parameters, compute and data grow. This means there are structural causes that scale alone cannot fix.

RAG (retrieval-augmented generation), which retrieves external documents and attaches them as evidence, is not a complete solution either. VOTE-RAG paper explains that if the retrieval results are themselves wrong, the generator is pulled along and errors compound, producing "hallucination on top of hallucination." What makes this topic hard is that even attached evidence can be wrong.

Core idea: the student who guesses on an exam

The analogy used in 'Why Language Models Hallucinate paper' is a student facing a hard exam question. Leaving it blank earns zero, but guessing might be right. So writing a plausible answer, even without knowing, is better for the score. The paper argues that language models are in the same position: when uncertain, they guess a plausible answer rather than say "I don't know."

This explanation has two steps. First, in pretraining, hallucination is not a mysterious phenomenon but arises from errors in binary classification (deciding right vs. wrong). If false statements cannot be distinguished from facts, natural statistical pressure alone produces hallucinations. Second, most evaluations score only whether the answer is correct. So models are optimized to be "good at tests," that is, to guess when uncertain.

This view also changes the remedy. The paper argues that adding one more hallucination-specific evaluation is not enough. Instead, it says the scoring of existing benchmarks that dominate leaderboards, which penalizes uncertain responses, must be changed, and calls this a socio-technical mitigation. The core of this view is that the answer to "why was it designed this way" lies not in model architecture but in the evaluation system.

How it arises and where to catch it

Breaking the path to hallucination into stages reveals where to intervene. ① During pretraining, as the model learns statistics by predicting the next token, hard-to-distinguish facts remain as errors. ② In evaluation and reward, scoring only correct answers rewards guessing. ③ At the moment of generation, the model's internal state determines whether it heads toward the correct answer or toward hallucination. ④ After output is produced, it can be filtered through detection and verification.

'Hallucination Basins paper' examines stage ③. It analyzes the Trajectory of Hidden Representations the model passes through as it generates tokens one by one, and argues that the latent space (the model's internal numerical space) contains task-specific "basins," depressions into which states are drawn. For factual questions, the basins for correct answers and hallucinations are relatively distinct, but for summarization or misconception-prone questions they often overlap. The study reports that using this geometry to slightly redirect generation (steering) can lower the probability of hallucination without retraining.

Detection at stage ④ has several approaches. The OOD perspective study paper treats next-token prediction as a classification problem and adapts out-of-distribution (OOD, input types not seen during training) detection techniques from computer vision to language models. The result is a training-free detector that judges from a single sample, with high accuracy on reasoning tasks as well. The small-model verification study paper splits an answer into sentences, then aggregates the probabilities that multiple SLMs (small language models) answer "Yes" given the retrieved context, to judge whether each sentence is supported by the evidence.

There are also ways to make generation less wrong in the first place. VOTE-RAG paper uses two stages: "retrieval voting," where multiple agents gather documents with diverse queries, and "response voting," where multiple agents answer independently and a majority vote decides. It needs no separate training, and every step can run in parallel.

How the field developed: three tracks of cause, detection and mitigation

The starting-point papers for this material are less a single lineage than three tracks that moved in parallel over the same period. The first track is explaining causes. 'Why Language Models Hallucinate paper' explained hallucination through statistics and the evaluation system, and 'Hallucination Basins paper' explained it through the model's internal geometry, noting that how easy it is to separate differs by task.

The second track is detection. The small-model verification study paper experimented on about 100 sets of questions, answers and contexts, and reported a 10% improvement in F1 score (the harmonic mean of precision and recall) for separating correct responses from hallucinations. The hallucination classification framework paper deliberately reproduced various types of hallucination through prompt engineering and analyzed them as clusters in embedding space. It found a correlation in which hallucinations with more severe information distortion lie farther from the cluster of correct answers, and concluded that even a simple classifier could tell the two apart. The OOD perspective study paper extended detection to reasoning tasks, where existing detectors had been weak.

The third track is mitigation. VOTE-RAG paper performed comparably to or better than more complex frameworks on 6 benchmarks. Because of its simple voting structure, it also avoids the risk of "problem drift," where the question shifts in a different direction. The neural diversity study paper proposed "neural diversity," which makes parallel Representations less similar to one another. Its implementation, ND-LoRA (parallel LoRA adapters combined with a regularization that reduces correlation), reduced hallucination by up to 25.6% and by 14.6% on average while maintaining general accuracy. It also reported an association in which a 0.1% increase in neural correlation comes with a 3.8% increase in hallucination, and called this a "third scaling axis" separate from parameters and data.

Where things are heading

The first trend is building the diagnosis "don't reward guessing" directly into training rewards. CalibraRL paper notes that reinforcement learning that rewards correct answers, penalizes wrong ones and also allows abstention (not answering) is already in use, but that fixed rewards cannot distinguish confident wrong answers from hesitant ones. So when the model answers, it is penalized more the more its confidence and correctness diverge; when it abstains, the reward is set by estimating how reliably the model answers that question. Because it reuses Rollouts already generated during training, no extra labels on "does the model know this answer" are needed. It reports higher answer precision and fewer hallucinations on three QA datasets.

The second trend is that the scope of hallucination has widened from text to actions. StateMirror paper calls it "hallucinated tool use" when a coding agent calls nonexistent paths or packages that are not installed. A study of 4,398 agent Trajectories found such hallucinations to be common, and recovering from them took a median of 3 extra interactions and 22,400 (22.4K) tokens. StateMirror intercepts actions at the harness boundary and checks them deterministically against the actual repository state before execution. On 2,508 actions it achieved a precision of 0.940 and an F1 of 0.817, 35.3% higher in F1 than the strongest LLM judge. Median detection latency dropped from 17.73 seconds to 1.50ms, and it used no tokens. With feedback received before execution, agents fixed more than 95% of hallucinated tool calls on their own.

The third trend is the measurement problem of "how far to trust detection scores." Mycelium paper points out that benchmarks differ greatly in how they define and measure hallucination, making scores hard to compare. It therefore organizes benchmarks along three axes: source (how the outputs and evidence were obtained), type (factuality failure or faithfulness-to-evidence failure), and unit (at what granularity labels are assigned and evaluated). It also showed that in span-level evaluation, even small differences in label boundaries cause large swings in scores.

Limitations and practical criteria

The most common mistake is to assume that 'adding RAG solves hallucination.' As VOTE-RAG paper points out, wrong retrieval results can actually make hallucination worse. It is safer to check retrieval quality separately and to add mechanisms such as diversifying search queries or comparing multiple answers.

Second, how well a technique works depends on the task. According to 'Hallucination Basins paper', internal signals make it easy to separate hallucinations on factual questions, but on summarization or questions containing misconceptions, correct answers and hallucinations overlap. The neural diversity study paper also reported that the right degree of diversity differs by task. Do not assume that a detector that works well on demo factual questions will also work well on summarization tasks.

Third, do not compare detection performance numbers at face value. As Mycelium paper showed, the same 'hallucination detection F1' means different things depending on what the benchmark counts as hallucination. Before adopting a technique, first decide what unit (sentence, span or whole answer) and type of 'hallucination' to use on your own business data, and re-measure against that standard.

Fourth, abstention and accuracy trade off against each other. Allowing 'I don't know' reduces hallucination, but too much of it increases useless abstentions. This is why CalibraRL paper adjusts the abstention reward according to per-question confidence. In practice, too, look at the abstention rate and the error rate together, and decide whether your service dislikes 'wrong answers' or 'no answer' more. The table below sorts the techniques covered in this article by whether they require retraining, external evidence or access to model internals. If you use models only through an API, it is practical to start with techniques that do not require internal access.

Further reading

If you are new to the topic, start with 'Why Language Models Hallucinate paper'. Once you take its view of hallucination as a problem of classification errors and evaluation practices, it becomes easier to sort out which stage the other papers target. For a single overview, the review that briefly summarizes the types, causes and mitigations of hallucination paper works well as an introduction.

If you want to know what happens inside the model, read 'Hallucination Basins paper' followed by the OOD-perspective study paper. If you run a RAG service, VOTE-RAG paper and the small-model verification study paper show training-free methods you can apply right away.

Three papers are enough for the latest trends: CalibraRL paper on reward design, StateMirror paper on agents, and Mycelium paper on evaluation methodology. Mycelium in particular explains how to read the numbers in other detection papers, so it is worth reading before you compare or adopt techniques.