Deep dives · 2026-10-21 · Advanced

Context engineering: beyond prompts, what to include and how much

Context engineering is the design work of choosing, trimming, folding and restoring what an agent on a long task shows the model at each step, and how much of it. Research has moved from fixed summarization rules toward learned compression and folding policies, reversible rendering, and approaches in which the model edits its own context.

Contents

Why it was needed: context that only accumulates
Core idea: pack light, but keep what you need retrievable
How it works: judge, trim, show again
How it developed: from fixed rules to learned policies
Where it is heading: reversible, inside the model, fitted to behavior
Limitations and open issues
Further study

Key points

On long tasks, if history is only ever appended, the context overflows, noise accumulates, costs rise and reasoning quality drops.
The central question is "Will this information be used in the next step?", and studies implement this judgment through summarization, folding, card substitution and pruning.
As results kept showing that policies that adapt to the situation beat fixed rules, a trend emerged of elevating context management into a tool, a learning target and a native model capability.
Wrongly deleted information is hard to recover and can also break the prompt cache, so reversible designs that preserve the original are drawing attention.
The same technique can help or hurt depending on an agent's behavioral tendencies, so techniques should be chosen after observing actual behavior.

Why it was needed: context that only accumulates

Where prompt engineering focused on "how to write a single request well," context engineering deals with "how to organize all the information the model sees across many steps." AI agents that search the web dozens of times or move around a code repository making fixes accumulate tool results, observations and thoughts at every step. Many agents simply append this history (append-only) and resend all of it to the model at each step. ReFold paper points out that, as a result, context and cost grow together as steps increase, until the context eventually exceeds the context window (the maximum length a model can read at once).

Length is not the only problem. The SLIM study paper, which analyzed long-horizon web search agents, identified the main causes of failure as existing frameworks piling up long, noisy content until they hit the context window or tool-call limit, or stopping too early. CAT paper, which covers software development agents, explains that this append-only or passive compression approach leads to context explosion, a gradual drift in meaning (semantic drift) and degraded reasoning quality.

There is also the "lost-in-the-middle" phenomenon, in which models tend to miss information placed in the middle. The ContextCurator study paper finds that noise accumulated from verbose environment outputs clouds reasoning over many rounds of interaction. In the end, even with a generous context window, the assumption that "more is better" does not hold. What to include, what to shrink and what to remove becomes a design problem.

Core idea: pack light, but keep what you need retrievable

Think of packing for a long trip. If you carry every receipt from the journey so far, your bag gets heavy and it becomes hard to find what you actually need. But if you throw things away at random, you lose documents you will need later. A good traveler puts only "things for the next leg of the itinerary" in the bag and leaves the rest in the hotel safe, to take out again when needed. CarryOn paper uses the phrase "Pack Light" for the same intuition.

The decision criterion in context engineering is likewise "value for future use." PAACE paper decides what to compress by looking at relevance to the next few tasks (next-k-task relevance) and the plan structure. CAT paper divides the workspace into three parts: a fixed task goal, compressed long-term memory, and recent interactions kept in their original form. It is a structure that shrinks old material and keeps recent material vivid.

So context engineering is less a skill of "writing" prompts well and more a skill of "allocating" a limited space well. The space is fixed, and the task goal, memory, recent records and tool descriptions compete for it.

Allocating a limited context

Context window

Task goalCompressed long-term memoryRecent interactionsOld records that only accumulated

Illustrative proportions showing a CAT-style workspace layout. Old records are shrunk into compressed memory, while recent interactions are kept as they are.

**Context Engineering: Architecting Information for AI Agents** · AI-generated with Gemini Notebook

How it works: judge, trim, show again

Overlaying multiple studies reveals a common loop. First, the results of the agent's actions and observations accumulate in the history. Second, at set points (regular intervals, milestones, budget overruns), the system judges what to keep. Third, it summarizes the parts it will not keep, folds them into one-line notes, or replaces them with short cards. Fourth, it shows this refined context to the model and receives the next action. Then the loop repeats.

The specific methods differ by study. SLIM paper separates search and page reading into distinct tools and periodically summarizes the Trajectory (the record of actions so far). Context-Folding paper branches off to work on a subtask, and when finished, folds the intermediate steps and keeps only a summary of the result. CADOC paper turns structured objects into short cards for retrieval, while keeping the originals so they can be retrieved exactly when needed.

In real tools, this loop sits inside a broader harness (Agent Harness, the execution framework that wraps a model and attaches tools, memory and rules). The terminal coding agent OPENDEV paper combines adaptive compression that gradually shrinks old observations, lazy tool discovery that finds and attaches tools only when needed, automatic memory that builds project knowledge across sessions, and event-driven system reminders that prevent instructions from fading as they get pushed back in the conversation (instruction fade-out). Splitting planning and execution into separate agents is another way to keep each one's context light.

Context management loop Starting from the history at top left, the context is refined moving right and then shown to the model, and the action results flow back into the history. The dashed line is an optional flow that preserves the original separately.

How it developed: from fixed rules to learned policies

Studies in the second half of 2025 refined "when and how to summarize." SLIM paper, with a simple structure alone, scored 56% on BrowseComp and 33% on HLE on an o3 base, 8 and 6 points higher than open-source frameworks respectively, with 4–6 times fewer tool calls. Context-Folding paper taught folding behavior through reinforcement learning (FoldGRPO), matching or outperforming ReAct-based agents with a 10 times smaller active context and clearly beating summary-based management. PAACE paper distilled a plan-aware compression model from a Teacher's successful demonstrations, raising accuracy while reducing maximum context length. CAT paper made context management a "tool" that the agent calls directly, and the resulting trained SWE-Compressor solved 57.6% on SWE-Bench-Verified.

In 2026, a clear direction emerged of "not sticking with a single strategy to the end." AgentSwing paper observes that a fixed strategy fits well in some states and poorly in others, so at each switch point it expands several management approaches in parallel, then looks ahead and picks the most promising branch. As a result, it matched or exceeded fixed approaches with up to 3 times fewer conversation turns. ContextCurator paper separates management from execution: a lightweight curator model trained with reinforcement learning cuts out noise, and a fixed large model does the task. On WebArena it raised Gemini-3.0-flash's success rate from 36.4% to 41.2% while cutting tokens from 47.4K to 43.3K, and a 7B curator achieved management performance at the GPT-4o level.

ContextEvo paper goes a step further and evolves the management policy itself. At key decision points, it reconstructs the context the model actually saw, finds failures caused by the context, and revises the policy. Starting from the open-source Pi-agent harness, it achieved results comparable to or better than harnesses such as Codex and OpenCode on three long-horizon task benchmarks. This trend can be summarized as "human-defined summarization rules → learned compression and folding → management policies that adapt to the environment."

Where it is heading: reversible, inside the model, fitted to behavior

The first direction is "reversible management." ReFold paper works only at the rendering layer, without a predictive model or training. It leaves the original history intact and compresses only when showing it to the model. Content already shown earlier is replaced with a short marker (stub), and turns the agent has declared finished are folded into one-line notes. Anything removed this way can be restored from the history in one step. It also rewrites in batches every few steps rather than at every step, so that prompt caching (a technique that reuses computation for requests with the same prefix) breaks less often. Across five benchmarks it cut tokens by up to 2.5 times without lowering success rates, and halved KV cache memory per session.

The second direction moves editing inside the model rather than outside it in the harness. NCE paper turns context-editing operations into a "grammar" of dedicated special tokens, teaching the model to edit its own context through language itself while generating an answer. The claim is that this generalizes better across diverse tasks than learning the tool-call code of a specific harness. On Terminal-Bench 2.0 and four SWE-bench-family benchmarks, it raised average success rates over the base models by 5.6 percentage points for LongCat-Flash-Lite-Sparse and 4.5 percentage points for Qwen3.6-35B-A3B, while reducing token usage by 36% and 15% respectively.

The third direction is behavioral research asking "who a technique suits." A study on data agents paper tested a semantic layer (a layer that describes the data), orchestration that splits work across multiple agents, and history compression (compaction) with 7 LLMs and 1,514 long-horizon tasks. The agents fell into types that investigate thoroughly, types that rush to answer without looking at the data, and types that give up early, and each technique translated into accuracy only for the type it suited. For the thorough type, it was actually harmful: the semantic layer made them stop inspecting directly, orchestration induced overconfidence and cost 5 percentage points, and compression made them repeat the same steps. Combining techniques produced less effect than the sum of each.

Map of context engineering research Arranged chronologically from top to bottom. Items in the same group are studies from a similar period; click a box to go to the original paper.

Limitations and open issues

The biggest risk is deleting the wrong thing. ReFold paper criticizes existing predictive methods for adding execution overhead through extra model calls, invalidating the prompt cache, and offering no guarantee that deleted content can be recovered. There is also an observation that compression harms "answering correctly every time, reliably" before it harms "being able to answer correctly." PAIR paper compared compressed and uncompressed runs from the same state side by side and showed that severe performance drops are concentrated at a few individual compression points.

When to forget depends on timing. RECAP paper reported cases where, with the same failure record and the same summary, compression at an early point failed while compression at a later point saved cost. In other words, it is hard to attach a fixed label of "this information is important / not important." As in the data agent study paper discussed above, effects can reverse depending on an agent's tendencies, so adopting a technique as-is because others found it good can backfire.

Care is also needed when reading the numbers. All performance and token figures in this article come from specific benchmark and model combinations, and each study uses different benchmarks and baselines. There is no guarantee the same gains will appear in real service environments. There is also a trade-off: learning-based methods require training a separate model, while rendering approaches need no training but make simpler judgments.

Deleting vs. reversible approaches

Predictive managementExtra models · rules · learned policies

Execution overhead from extra model calls
Prompt cache may break
No guarantee deleted content can be recovered

Reversible renderingReFold · CADOC approach

Original history preserved intact
Rewrites in batches every few steps to reuse the cache
Wrong deletions restored in one step

공통 Both aim to shrink the context shown to the model

A side-by-side comparison of the two design directions covered in the text.

Further study

If you are new to this, start with SLIM paper. It uses an error taxonomy to show why long-horizon search agents fail because of context, and its solution is simple, making it a good way to grasp the problem. Next, if you want a feel for the design of real tools, read OPENDEV paper. You can see how compression, memory, system reminders and lazy tool loading fit together inside a single harness.

If you are curious about learning-based approaches, read Context-Folding paper and CAT paper together. One teaches folding through reinforcement learning, and the other teaches context management as tool calls. AgentSwing paper and ContextEvo paper illustrate well the view of adapting management policies to the situation.

For recent trends, look at ReFold paper for reversible design and the cache problem, NCE paper for editing as a native model capability, and the data agent behavior study paper for "criteria for choosing techniques." Learning the related terms context window, prompt caching, harness and agent memory first will make reading easier.

References

Foundational papers

Recent research

Written by AI from the listed references and reviewed. Please check the original sources for details.