Harness adapters: fitting the 'work environment' instead of the model

Why it was needed: swap only the model, and the harness no longer fits

An agent is not a language model acting alone. The model is wrapped in an execution framework that decides what to observe, how to call tools, how to actually execute actions, how to return failure messages and when to stop. This is called the harness (agent harness). The Life-Harness study paper pointed out that an agent is shaped not only by the model but also by the runtime harness, which handles observation, tool use, action execution, feedback interpretation and control of the trajectory (the record of actions).

The problem is that harnesses are usually designed around a specific frontier model. According to the SLM harness optimization study paper, frontier LLM agents have high inference costs, which makes them hard to deploy at scale. But when an SLM (small language model) is plugged directly into a harness built for a frontier model to cut costs, performance usually falls short of expectations. The model became smaller while the harness stayed the same.

Until now, agents have mainly been adapted to tasks by changing model parameters through fine-tuning or reinforcement learning. Life-Harness paper observed that in domains with clear rules and fixed (deterministic) outcomes, many failures come not from a lack of model capability but from a mismatch in the interface between the model and the environment. In that case, what needs fixing is not the model but the interface, that is, the harness. Harness adapters grew out of this observation.

Core idea: don't change the worker, fit the workbench

When a new worker joins a factory, the company may retrain them, but it is often faster to adjust the jigs (work-holding fixtures) and work instructions to suit that person's hands. Harness adapters follow the same idea. Model weights stay fixed, and the workbench the model works at (instructions, tools, execution order) is refined to suit that model and that task.

The SLM harness optimization study paper explains the basis for this idea as follows. Much of a task's difficulty does not differ from case to case but appears in common across many cases. If a difficulty is shared, there is no need to make the model work it out alone every time; it can be put into the harness once, in the form of custom instructions, tools and an orchestration loop. The researchers described this as 'lifting' difficulty from the model into the harness.

The trade-off is also clear. Fine-tuning builds capability into the model, so it is retained to some degree when moved to a different harness, but it requires training cost and data, and it cannot be used with API models whose weights are not released. Harness adaptation applies immediately without training and remains in a human-readable form, but it tends to be tied to that particular model–task combination. So harness adapters are less about finding 'the best harness for any model' and more about finding 'the harness that fits this model'.

How it works: from failure to harness fix

Automated harness adaptation generally runs in the following order. First, training tasks are run multiple times with a fixed model to collect trajectories. Second, failed trajectories are examined and sorted into failure types. The SLM harness optimization study paper built a framework pairing agent failure types with harness adaptation strategies, and had a meta-agent (an agent that fixes other agents) automatically find effective fixes from failed trajectories.

Third, candidate fixes for the failures are created. What gets fixed is divided slightly differently from study to study. The SLM harness optimization study paper modified instructions, tools and the orchestration loop. Life-Harness paper turned recurring interaction failures into four types of reusable intervention: 'environment contracts', which spell out the agreements with the environment; 'procedural skills', which contain procedures; 'action realization', which turns actions stated by the model into actual execution; and 'trajectory regulation', which keeps the execution flow from going off track.

Fourth, candidate fixes are evaluated and only those that improve results are kept. Fifth, the resulting harness is frozen during evaluation and used on unseen tasks. An example is Life-Harness paper, which evolved the harness using training trajectories and then froze it for evaluation. The biggest difference from fine-tuning is that the model weights and the evaluation environment do not change at any point.

How the field developed: what the founding studies introduced

Life-Harness paper took a clear position: 'fix the interface, not the model'. It was evaluated with 18 open-weight models across 8 environments from AgentBench, τ-bench, τ²-bench and AppWorld. With Qwen3-4B-Instruct, its average accuracy over the 7 shared environments was 28.7%p and 7.5%p higher than the existing automated harness methods Meta-Harness and AHE, respectively. It used 8.5% and 68.4% fewer tokens for optimization, respectively. When the evolved harness was attached to frontier models such as GPT 5.5, Opus 4.8 and Gemini 3.1 Pro, performance also rose by an average of 3.5%p.

The SLM harness optimization study paper addressed cost directly. Of 21 pairs combining 7 business-style agent tasks with 3 SLM families, the optimized harness significantly improved performance in 16 pairs, and in 7 pairs it closed the performance gap between the SLM and the LLM. The best SLM agent reached 89.7% of LLM performance at 4% of the cost. The researchers concluded that for many routine tasks, an SLM paired with an adapted harness can match LLM performance at 90% lower cost.

Some studies narrowed the target of adaptation from the whole harness to a component within it. RASO paper focused on agent skills, reusable natural-language documents that guide an agent to work well in a specific harness. Although millions of public skills have accumulated, existing methods revised skills using only expensive rollouts (actual executions). RASO proposed 'Cross-Harness Adaptation', which retrieves relevant knowledge from external skill collections and transfers it to fit the target task and harness. It is split into a stage that creates initial skills without rollouts (RASI) and a stage that retrieves and revises based on execution feedback (RASU), and across 4 benchmarks and 2 models it consistently outperformed baselines without retrieval-based initialization and updates.

Harness adaptation has also expanded into safety and training. HASTE paper evolves the harness from sparse evidence alone, such as a short description in a threat report or a few attack examples. It pits a side that writes safety specifications against a side that generates attack cases, and across multiple models and attack types it lowered attack success rates while preserving utility on normal tasks. ScienceBuddy paper linked harness evolution and model reinforcement learning in a dual loop. The inner loop fixes the harness while keeping the model frozen, and the outer loop trains the model under the improved harness. The harness changes the training experience, and the trained model in turn creates new opportunities for harness adaptation.

Why fit changes: model–harness interaction

Why harness adapters deserve separate attention is made clear by the model–harness fit study paper. The researchers compared 66 configurations on TUA-Bench, ALE-CLI and Terminal-Bench 4, combining 4 configurable harnesses (OpenHands, DeepSeek Harness, PI, openJiuwen), 5 models, and vendor-native pairings such as Codex-GPT and Claude Code-Claude. To do this, they built adapters connecting multiple models to multiple harnesses, and released those adapters, the evaluation code and 6,204 scored trajectories.

The result was that the phrase 'a good model' does not hold up without the harness. On Terminal-Bench 4, Claude led GPT by 7.94 points in OpenHands but trailed it by 30.16 points in PI. For 4 of the 5 models, the best harness changed when the benchmark changed. A model maker's own harness was not always the best, and spending more did not raise scores. On Terminal-Bench 4, GPT scored higher in PI than in DeepSeek Harness at less than a quarter of the cost per task.

The researchers compared trajectories on the same tasks side by side to find the reasons. Because repair is almost always initiated by the model itself, whether the harness returns failures in a form the model can use largely determines success or failure. GPT did best in the minimal PI, while Kimi, which often produces malformed tool calls, scored highest in openJiuwen on all three benchmarks. The study concluded that models, harnesses and tasks must be evaluated together. This is why designing a harness adapter starts with the question 'what mistakes does this model make often?'

Where things are heading

The first direction is assembling a harness for each task instead of fixing a single harness. STITCH paper argued that each harness mechanism (context gathering, tool calling, verification, state preservation, termination) can help on some tasks and become a burden or distraction on others, so a single fixed harness cannot be optimal. But generating new harness code for every task piles up generation and debugging costs and execution risk. So it mines reusable components ('Harness Primitives') with clear scopes and composition rules from failed trajectories, and at test time selects and assembles only the components that match the task information. Task success rates rose by up to 12 points over fixed harnesses, performance improved as the component library grew, and it also outperformed human-designed harnesses such as Codex CLI.

The second direction is evolving the evaluation environment along with the harness. HERA paper addressed 'agent abstention', the ability to recognize unsolvable tasks and stop. To address the problem that optimizing a harness only on a fixed task set leaves it weak against unseen failure types, it deliberately altered solvable tasks to create tasks that require abstention, and kept generating new environments targeting earlier weaknesses. The evolved harness raised abstention accuracy from 61.7% to 83.3% and the completion rate on feasible tasks from 68.3% to 76.7%. Transferred without model-specific optimization to 19 other LLMs, the harness raised abstention accuracy by an average of 15.3%p, and let small models match large-model performance while cutting costs by about 85%.

The third direction is making continual learning during operation safe. VIGIL paper improved the harness for spreadsheet question answering using ground-truth labels that arrive later, but kept the base model, prompts and retriever fixed and updated only a contract calibrator and a column selector. This narrow-scope update alone raised accuracy from a static baseline of 75.8% to 82.0–82.4%, and on a stricter split, average held-out accuracy rose from 80.3% to 86.7%. Updating only the calibrator raised accuracy by 5.9 points, close to the 6.4 points of the full update. It is a design that keeps 'how much of the harness to allow to change' narrow.

Limitations, open issues and practical criteria

When to use it. According to the analysis in the SLM harness optimization study paper, harness adaptation works best for tasks with repetitive workflows and SLMs with sufficient base capability. Of 21 pairs, 16 showed significant performance gains; for the rest, the harness alone was not enough. A harness only eases shared difficulties; it does not create capabilities the model lacks. For open-ended problems where requirements vary widely from task to task, fine-tuning or a larger model may be the right choice.

Common mistake 1: believing a fix is safe because it passed the gate. In VIGIL paper, 3 of the 26 updates approved by the gate lowered accuracy on held-out data compared with the previous version. The researchers noted that no regression on the replay buffer (a check that re-runs past data) does not mean no regression on unseen data. Harness fixes can overfit to training trajectories, so they must be checked separately on a held-out set not used for adaptation.

Common mistake 2: carrying conclusions from one combination directly over to another. As the model–harness fit study paper shows, model rankings and the best harness reverse depending on the harness and benchmark. If you change the model, re-evaluate the harness, and do not assume that the vendor's harness or a more expensive configuration is best. On the other hand, there are cases like Life-Harness paper and HERA paper where gains persist when transferred to other models, so confirm transfer effects by measurement, not assumption.

Common mistake 3: keeping only the result numbers and not the evidence. The agent safety evaluation study paper pointed out that even with the same task success rate or attack success rate, the underlying level of evidence can differ greatly. This study built a cross-harness adapter that takes execution records from multiple harnesses and issues an 'evidence sufficiency card' for each decision. For 4 inputs that read as the same result on the surface, sufficiency scores ranged from 0.458 to 0.833, and none of the scored records met the preconditions for re-execution. The more automatically a harness evolves, the more important it becomes to keep records from which you can reconstruct what changed and why. Safety rules also live in the harness, so when new attacks appear, a loop that re-verifies them with attack cases, as in HASTE paper, is needed.

Further study

If you are new to the topic, start with the SLM harness optimization study paper. It lays out from a practical business perspective why difficulty can be moved into the harness, how failure types are paired with adaptation strategies, and which tasks it works well for. Then, looking at the classification of interventions in Life-Harness paper (environment contracts, procedural skills, action realization and trajectory regulation) gives a sense of which parts of the harness can be fixed.

For practitioners choosing models, the model–harness fit study paper is essential. You can also run your own comparisons with the released adapters and 6,204 trajectories. If you want to use a different harness for each task, continue with STITCH paper; if you want reuse at the skill level, continue with RASO paper.

If you are responsible for operations and safety, read in order VIGIL's paper gate design and its limits, HERA's paper abstention evaluation, HASTE's paper attack-driven evolution, and the evidence reconstruction metrics for agent safety evaluation paper. For the big picture of running harness and model training together, the dual-loop structure of ScienceBuddy paper is a good illustration.