Deep dives · 2026-11-18 · Basic · By bombong
Agent evaluation: trajectory scoring and cross-version regression detection
If agents are scored only on whether the final answer was correct, runs that passed by luck cannot be told apart from runs that solved the task properly. This note covers how to score the execution process (the trajectory), and what to hold fixed and compare to catch silent performance drops (regressions) when changing versions.
Contents
- Why it was needed: the limits of a single pass/fail bit
- Core idea: grade the working, not just the answer sheet
- How it works: record → phase labels → comparison with reference paths
- How it developed: several branches of process-aware evaluation
- Cross-version regression detection: what to hold fixed and what to compare
- Where it is heading: what to show the grader
- Limitations and open issues: what to consider in practice
- Further reading
- Outcome-only scoring treats a run that passed after trial and error the same as one that solved the task methodically. In a study of SWE agents, 10.7% of passing trajectories were such 'Lucky Passes'.
- Trajectory scoring splits actions into phases such as exploration, implementation and verification, and compares them with reference paths that solved the same task properly, turning waste and ordering errors into scores.
- To catch cross-version regressions, first check whether the evaluation harness itself is stable across repeated measurements and whether runs can be reproduced exactly (random seeds, hidden state).
- What you show an LLM grader changes its verdict. Grading that sees the full execution record produced fewer false alarms than grading that sees only text, and hiding the record through summarization or delegation made verdicts swing widely.
- Functional evaluation and security red-teaming find different failures, so one should not substitute for the other.
Why it was needed: the limits of a single pass/fail bit
Unlike a chatbot that answers once and stops, an AI agent solves a task by thinking several times, calling tools and looking at the results. The ordered record of thoughts, tool calls and observations left by this process is called a trajectory. Yet most agent evaluation so far has looked only at the final outcome. For a coding agent, a single bit—'did the final patch pass the tests?'—was the entire score.
AgentLens paper points out that this approach treats a methodical run and a run of repeated trial and error the same, and shows that the two are in fact not the same. It analyzed 2,614 OpenHands trajectories in which 8 model backends worked on 60 SWE-bench Verified tasks, and separately collected 1,815 trajectories from the 47 tasks for which evaluation criteria could be built. Among these, 10.7% of passing trajectories showed fix-then-break cycles, unjustified retries, skipped verification, or a jumbled order of exploration, implementation and verification. The researchers called this a Lucky Pass.
Similar problems appeared in other areas. M3-BENCH paper, which examined agents' social behavior in mixed-motive games, reported that outcome-only evaluation misses large differences in social ability. The safety evaluation framework ForesightSafety-SAGE paper also started from the point that existing evaluations, which rely on handwritten scenarios, fixed prompts and judgments of final outputs, struggle to catch the varied risks that arise during execution.
Core idea: grade the working, not just the answer sheet
A math exam analogy makes this easy to understand. If only answers are graded, a student who guessed correctly and a student who solved the problem properly get the same score. On a single exam this may not matter much, but the student who guessed is more likely to get it wrong on the next one. Looking at the working as well reveals this risk in advance.
Trajectory scoring does exactly this. It looks at what the agent examined first, whether it checked after making a fix, and whether it repeated the same attempt pointlessly. It does not discard outcome scoring; it adds one more axis—process—on top of it. The trade-off is also clear. Outcome scoring is cheap and has clear criteria. Process scoring, by contrast, requires defining what a 'good process' is, and building and maintaining that definition costs effort.
From a version-management perspective, this difference matters even more. If you change the model or prompt and the pass rate stays the same, outcome scoring alone leads you to conclude 'no change'. But if the share of Lucky Passes has grown in the meantime, quality may already have started to decline. Process metrics are needed to see such hidden regressions.
Outcome grading vs. trajectory grading
Outcome gradingLooks only at final answers and tests
- Cheap, with clear criteria
- Cannot identify lucky passes
- Regressions hidden when pass rates match
Trajectory gradingAlso looks at the solution process
- Catches waste, skipped verification, wrong ordering
- Reference paths must be built and maintained
- Reveals hidden quality loss
공통 Both need the same task set and fixed run conditions to be used for version comparison
How it works: record → phase labels → comparison with reference paths
Following the AgentLens paper approach shows the basic flow of trajectory scoring. The first step is to collect execution records. Have multiple models, or multiple runs, solve the same task to accumulate trajectories.
The second step is to label each action with 'what it was meant to do'. AgentLens classifies each action as exploration, implementation, verification or orchestration. It considers not only which tool was used but also the context of earlier records. Opening the same file may be exploration when the agent is first learning the structure, but verification when it is checking results right after a change.
The third step is comparison with reference paths. AgentLens merges multiple passing solutions to the same task into a per-task reference called a PTA (Prefix Tree Acceptor, a structure that merges the common prefixes of multiple paths into a tree). The key is not fixing a single correct path but using 'the route good solutions generally took' as the reference. It assigns a quality score by examining where a new trajectory diverged from this reference (divergence points) and where waste occurred (waste signals). Using this score, passing trajectories are divided into three grades—Lucky, Solid and Ideal—and Lucky Passes are further broken down into five recurring causes.
For version comparison, run this same flow on both the previous and new versions, then look at outcome scores and process scores side by side. When user input is needed, as with conversational agents, a user simulator generates it. How well the simulator imitates a diverse range of users strongly affects evaluation results, as the next section discusses.
How it developed: several branches of process-aware evaluation
The studies this material starts from all appeared in 2026. Rather than building on one another, they each tackled the same concern—'outcomes alone are not enough'—from different directions.
M3-BENCH paper split evaluation into three perspectives across 24 mixed-motive games: behavioral trajectory analysis (BTA), reasoning process analysis (RPA) and conversation content analysis (CCA). Comparing 11 frontier LLMs with a human baseline, it found that reasoning models showed an 'overthink-undercommunicate' pattern: they think deeply internally but fail to turn that into effective dialogue. Safety-related risks that outcome metrics alone hide, such as appearing to cooperate while reasoning opportunistically underneath, also surfaced only when the perspectives were separated.
AgentLens paper applied process scoring to coding agents, defined the Lucky Pass, and built AgentLens-Bench, a dataset of 1,815 trajectories annotated with quality scores, waste signals and divergence points. ForesightSafety-SAGE paper automated everything from scenario generation to scoring on the safety side. It created 1,072 measurable scenarios across five risk dimensions and evaluated 12 agents; the average attack success rate (ASR) was 47.1%, and several models exceeded 70%. Based on these results, the researchers concluded that executable, process-level evaluation is important.
Some studies addressed evaluation cost and input diversity. ADWM paper is an off-policy evaluation method that estimates the performance of a new agent policy using only previously collected trajectories, without running it in the real environment. It imitates how the environment will respond with a diffusion-based world model, and has the agent under evaluation guide generation at each step so that the simulated trajectories reflect that agent's decision-making. A study on three-level persona vectors paper pointed out that one-line role descriptions such as 'You are an angry customer' produce nearly identical conversations regardless of scenario. It therefore built 23-dimensional personas made up of 6 demographic traits, 12 behavioral traits and 5 emotional states, and showed across 64,698 multi-turn conversations that agents' goal completion rates varied by up to 15.8 percentage points depending on the persona. In other words, a monotonous user simulator makes for monotonous evaluation.
Cross-version regression detection: what to hold fixed and what to compare
A regression is when a new version can no longer do something an earlier version did well. The basic principle for catching it is ordinary: run the previous and new versions on the same task set, and look at what changed per task rather than just the average. With agents, however, even creating 'the same conditions' is hard. When a difference appears, you have to determine whether it comes from the version or from instability in the evaluation environment.
First, the evaluation harness (the framework that runs and scores the agent) must be stable. A study comparing evaluation frameworks for enterprise analytics agents paper first measured harness stability by repeatedly scoring fixed execution records. 97% of configurations met the stability threshold. Doing this check first is what lets you separate score differences between versions from grader noise.
Second, runs must be exactly reproducible. A study of replay fidelity in AppWorld paper examined 71 failed runs through 284 isolated re-executions. Reconstructing only the declared configuration matched the original run in just 42 of 71 cases. Resetting the random number generator at every interaction raised this to 68. The 26-case gap between the two approaches shrank to 4 after database and oracle checks. Yet there were also 4 cases where outputs and verdicts were identical but the state actually left in the database differed. In some cases, identical API calls returned the same response while modifying different rows. This means you cannot conclude that two runs were 'the same' by comparing only the visible output.
Third, compare outcome metrics and process metrics together. AgentLens paper compared Lucky rates across 8 model backends. If switching models can change the Lucky rate while the pass rate stays the same, a regression test without process metrics will miss that change. Where running a new version in the real environment is costly or risky, methods like ADWM paper, which estimate a new policy offline from already-collected trajectories, are also being studied.
Runs accepted as the "same run" by replay method
Where it is heading: what to show the grader
The focus of recent research is moving a step beyond 'look at the process' to 'in what form should the process record be given to the grader?' The comparison study of enterprise analytics agents paper compared LLM grading that sees the full execution record, text-only LLM grading combined with red-teaming, and deterministic rule-based (heuristic) grading. On 72 test cases with known answers, execution-record-based grading had the best balance of precision and recall (F1=0.90). Text-only grading had the same recall but five times as many false alarms. In the stage of searching for not-yet-known risks, 24 new failures were found, and not a single failure found by functional testing overlapped with those found by security red-teaming. The researchers recommended a two-framework strategy: execution-record-based grading for functional evaluation and adversarial red-teaming for security validation.
The Verification Transport study paper measured whether an LLM verifier's verdict shifts when the same run is presented differently. In real systems, composite tools that bundle several tools, delegation of work to subagents, and summarization of long records (compaction) change the evidence a verifier can see. Even with the task, trajectory, outcome, ground truth, rubric and verifier all held fixed and only the presentation changed, the verdict failure rate reached about 31% under macro abstraction and delegation. Immediate-verdict approaches produced wrong verdicts more often. Evidence-grounded or deliberative approaches tended to withhold a verdict when information was insufficient. So a lower failure rate does not necessarily mean more reliability. Restoring the run's provenance information recovered 85.5% of failures.
The implication for regression detection is clear. If a new version changes the agent architecture so that delegation or record summarization increases, scoring results can change even when the agent's actual behavior is the same. In version comparisons, you must hold fixed not only the agent but also the form of the record shown to the grader.
Limitations and open issues: what to consider in practice
Process grading needs a reference path. Even AgentLens paper could build references only for 47 tasks that had enough passing trajectories. For new or rare tasks with few correct solutions, it is hard to define what a "good process" is. Also, differing from the reference path is not always bad. A novel but better solution risks being penalized, so it is safer to add a step where people review samples when a divergence point appears.
LLM-as-a-Judge, which uses an LLM as the grader, is convenient, but the grader itself needs to be evaluated. As the two studies above paperpaper show, verdicts change depending on the amount and form of information the grader sees. There are three common mistakes in practice: passing only the final answer text to the grader, changing how logs are summarized and still comparing against earlier scores as if nothing changed, and not checking whether the grader is stable across repeated measurements.
There are pitfalls on the input side too. If the user simulator's personas are monotonous, differences between versions may not show up, or, conversely, regressions that occur only for certain user types are missed. In the persona vector study paper, goal completion rates differed by up to 15.8 percentage points depending on the persona, which suggests that the user distribution in an evaluation set should be versioned like the agent itself.
In summary, here is when to use what. Outcome grading alone is enough when tasks are short and correctness is easy to verify. Add process grading when tasks are long and use multiple tools, or when runs that "passed but look shaky" lead to operating costs and risks. Do not substitute capability evaluation for safety and permission risks; set up separate red teaming for them.
Agent regression testing checklist
Fixed conditions
- Do you use the same task set and user persona distribution?
- Can you reproduce runs by controlling randomness and external state?
- Do you compare not only outputs but also the state left in storage?
Grader
- Are results stable when grading is repeated?
- Does the grader see the full run log, not just the final answer?
- Is the log summarization and delegation method the same as in the previous version?
Comparison metrics
- Do you look at process scores (such as the Lucky ratio) alongside pass rates?
- Do you check per-task changes rather than averages?
- Do you check security risks separately with red teaming?
Further reading
If you are new to this, start with AgentLens paper. It contains the backbone of trajectory grading in one paper: the concrete phenomenon of the Lucky Pass, action-step labeling and building the reference path (PTA). Next, M3-BENCH paper shows how splitting the process into perspectives such as action, reasoning and dialogue works outside of coding.
If you plan to build regression tests yourself, read the replay fidelity study paper together with the evaluation framework comparison study paper. The former shows how tricky it is to "recreate the same run"; the latter covers harness stability checks and criteria for choosing a grading method. If you use an LLM grader, check Verification Transport paper for how the form of the logs shown to the grader affects results.
Good next reads are the three-level persona vector study paper for input design for conversational agents, ADWM paper for offline evaluation of agents that are hard to run in real environments, and ForesightSafety-SAGE paper for automatic generation of safety scenarios.
Related terms
Evals LLM-as-a-Judge Harness LLM observability Red teaming Benchmark A/B testing AI agent
References
Foundational papers
- AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation (2026)
- M3-BENCH: Process-Aware Evaluation of LLM Agents' Social Behaviors in Mixed-Motive Games (2026)
- ForesightSafety-SAGE:A Fully Automated Scenario Generation and Safety Evaluation Framework for LLM Agents (2026)
- Autoregressive Diffusion World Models for Off-Policy Evaluation of LLM Agents (2026)
- A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation (2026)
Recent research
- Comparative Analysis of Agent Evaluation Frameworks: Stability, Detection, and Discovery in Enterprise Analytics Agents (2026)
- Replay Fidelity in LLM Agent Evaluation: Randomness, Hidden State, and Test Oracles in AppWorld (2026)
- Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation (2026)
Written by AI from the listed references and reviewed. Please check the original sources for details.