Dreaming: how AI learns "offline"

Why it was needed: the limits of learning only while awake

Ordinary learning happens in a single step: "see data, update the weights on the spot." This approach has three gaps. First, memories interfere with one another as they accumulate. The Hopfield network, a representative model of associative memory, stores patterns using the Hebbian rule (a rule that strengthens connections between neurons that fire together), and the clipped Hopfield study paper explains that storing too many patterns causes catastrophic forgetting, in which none of them can be recalled.

Second, real experience is expensive. Having a robot try grasping objects or flying a drone over a site to collect video takes time and human labor. Third, experience accumulates but is never organized. Even if a coding agent completes many tasks, it repeats the same mistakes unless those records are turned into a form usable for the next task.

Dreaming offers the same answer to all three problems: set aside separate time with no interaction with the real world, and during it have the model reprocess signals it generates itself or records it has stored. The name comes from human sleep and dreams, but in practice it is an engineering design that decides "when, using what material, and in which direction to update the model."

Core idea: experience by day, organize by night

As an analogy, it resembles an athlete's day. During the day they play real games; at night they replay the game footage and picture the next game in their head. They erase bad habits seen in the footage and, in their imagination, practice situations that are hard to try for real. In dreaming, "day" is the phase that receives real data, and "night" is the phase in which the model generates or replays signals internally.

Across the source studies, what the dream phase does falls into four broad types. The erasing type deliberately makes the model remember patterns from dreams less, reducing interference (the Hopfield line). The rehearsal type learns behavior from imagined experience inside a world model (a model that predicts how the world will change). The prediction type has the model guess future sensations during training to obtain better representations (robot touch). The organizing type extracts reusable knowledge from accumulated records and organizes it (LLM agents).

The design choice all four share is a trade-off: "use computation in place of real experience." The multilayer associative memory study paper shows this theoretically, reporting that a phase diagram with data disorder and "sleep time" as its two axes reveals a trade-off between data and computation. In other words, the scarcer or noisier the data, the more computation can be spent on offline organization to compensate.

How it works: the wake–dream loop

A dreaming system generally runs in the following order. 1) In the wake phase, it receives real data or experience. 2) It accumulates this in memory (weights, an experience store, a record store). 3) In the dream phase, it generates signals based on that memory. These can be random patterns, imagined rollouts unrolled by a world model, or predictions of future touch. 4) It filters or labels the generated signals. 5) It updates the model with the result: erasing, strengthening, or organizing separately. Then it returns to 1).

The erasing type works like this. The states a Hopfield network produces at random during the dream phase are close to "spurious memories" formed by mixing stored patterns. Learning them in reverse (unlearning) reduces the interference that many patterns create in common. The multilayer associative memory study paper states this more precisely: dreams pick out and weaken interference directions with large eigenvalues in the data correlation matrix, that is, directions where many patterns overlap and resonate strongly, while preserving the signal itself.

The rehearsal and prediction types go in the opposite direction. Instead of erasing, they use the future generated by the model as training material. Here step 4) is crucial. Because dreams can be wrong, they come with safeguards: using only verified records (Skill Dreaming), labeling dreams and isolating them with a time delay (Dream Layer), or having people select them (the world model data study).

Development: from associative memory to agents

On the memory theory side, three studies addressed the "function" of dreams. A computational model study paper showed in simulation that randomly activated signals in the hippocampus alone can produce learning and memory consolidation, and viewed dreams as a continuation of daytime processing. This is a rebuttal to the claim that "if dream content is random, it is useless." The clipped Hopfield study paper started from a problem: clipping, which caps connection strength, eliminates catastrophic forgetting but keeps only recent memories and greatly reduces capacity. Following a proposal by Hopfield, Feinstein and Palmer, alternating learning with a dream phase that generates random patterns and unlearns them increased storage capacity. The multilayer associative memory model DLAM paper put dreaming and inter-layer connections (learning that pairs different senses) into a single energy function, and with both present it recalled memories even in regions that neither could reach alone. Given a cue mixing several senses, it also separated out one original pattern per layer.

On the world model and robotics side, dreaming becomes "practicing through imagination." The probabilistic dreaming study paper improved the Dreamer model, which learns from imagined experience, so that it explores multiple latent states in parallel and maintains mutually exclusive hypotheses about the future separately. In the MPE SimpleTag environment it scored 4.5% higher than the baseline Dreamer, with 28% lower variance in episode reward. Tactile dreaming HTD paper had a humanoid robot, instead of only imitating actions, also predict future hand joint forces and latent representations of future touch. The targets come from a separate encoder updated slowly by an exponential moving average, so no separate tactile pre-training stage is needed. On five contact-rich real-world tasks, average success rate rose by a relative 90.9% over the stronger task-specific baseline, and the authors also reported that predicting touch in latent space was more effective than predicting raw signals.

On the LLM agent side, dreaming turns into "organizing experience" and "controlled hallucination." Skill Dreaming paper extracts actionable knowledge from verified PRs (code change requests) in a repository's PR history to build a repository-specific skill library, and refines the skills each time a new PR arrives. These skills guide coding agents not only in implementing features but also in maintaining tests, documentation, compatibility and repository conventions. Dream Layer paper approaches from the opposite direction: it loosens the logic-checking module and raises the sampling temperature to deliberately produce strange stories (for example, a travel story in which the currencies don't match), and uses them as data for safety testing on rare events and edge cases. It also proposed a shared dream repository collecting de-identified, abstract interaction templates.

Where it is heading: dreams as a data factory

Recent research is moving toward using world model dreams as a source that replaces real data. The world model data study paper fed wildlife drone footage as seeds into a 7B-parameter world model to generate new footage with the same sensor, altitude and environment, and used the selected frames to more than double the training set. On unseen footage from the same drone, mAP@50–95 rose 4.3% when the generated data was added to real data and 10.3% when it replaced real data, and by up to 33.5% on an urban benchmark. However, experts were estimated to have discarded about 87.5% of the generated footage. The cost was about $1,100 in GPU time, compared with more than $3,200 in field labor alone for a less useful real training set. The authors summarized this as data cost shifting from fieldwork to computation.

The problem of sustaining dreams over long stretches is also being addressed. A context compression study paper framed a world model's inability to consistently redraw a place when it revisits it as a "context retrieval" problem. Just as agent harnesses such as Claude Code select the context to send to an LLM, the authors attached a harness that compresses context for the world model, and without retraining it recovered rooms located up to 4x beyond the context window. By contrast, models using sliding window attention lost most of their context retrieval ability while dreaming.

Research has also appeared on how "realistic" dreams are. A butterfly dream study paper showed a fruit fly visual system model, trained only on optical flow estimation, real footage and footage continued by three small world models. In the trained observer's responses, all nine dreams were distinguishable from reality (AUC 0.85–0.99), while in the same network before training, the dreams of the trained world models were much less distinguishable (0.58–0.75). This means that even when pixel error is small, the flaws of dreams remain in temporal statistics, and it shows why quality checks are needed when dreams are used as training material.

Limitations and open issues: when to use it and when to avoid it

The first issue is dream quality. Most generated footage is discarded paper, and a trained observer easily tells dreams from reality paper. If unfiltered dreams are mixed directly into training, the model learns even the flaws peculiar to dreams. So in practice, "what to filter with" should be decided before "how many dreams to generate." This is also why Skill Dreaming paper uses only verified PRs as material.

The second is the boundary between dreams and reality. Dream Layer paper draws a distinction: unlabeled hallucinations produced online are still bugs, and only hallucinations that are scoped, labeled and used after a time delay become a resource for synthetic scenarios. To this end it combined three controls: strict abstraction, time delay and volatile memory. If provenance labels are not left on dream data, it later becomes mixed with real records irreversibly.

The third is the shift in cost. Dreaming saves data at the cost of computation paperpaper. If real data is cheap and plentiful, there is little reason to dream. Conversely, it is most effective when field collection is expensive, or when data that is hard to collect in reality is needed, such as rare events or dangerous situations. If the world model itself is weak, its dreams are weak too, so its ability to maintain context paper should be checked first.

One common mistake is confusing names. Paper searches for "Dream" also return results such as diffusion language models, delay-tolerant optimization methods and the Mori dream space from algebraic geometry, but these are unrelated to the dreaming covered here.

Further study

If you want to start with the principles, begin with the clipped Hopfield study paper. It gives the most concise treatment of the question "why does forgetting random patterns improve memory?" If you are comfortable with math, move on to DLAM paper to see the principle of suppressing interference modes and the data–computation trade-off. The computational model study paper covers the cognitive science background on the function of dreams.

If you are interested in reinforcement learning and robotics, look at probabilistic dreaming paper for the structure of imagined learning inside a world model, and tactile dreaming HTD paper for using future sensation prediction as an auxiliary task. For the practical cost perspective, the world model data study paper is the most concrete.

For developers building LLM agents, read in order: Skill Dreaming paper for the flow of organizing records into skills, Dream Layer paper for the controls that turn hallucination into a controlled resource, and the context compression study paper for managing long dreams from a harness perspective.