Paper review · October 28, 2026
Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents
Instead of collecting successful trajectories, the method deliberately builds the multi-step dependency structures that are missing from the data and trains only on actions that can be justified by information visible to the agent. This improved tool-use performance across all five Qwen3 sizes.
Problem
Training data for tool-use agents is usually filtered by whether the task succeeded. But even successful trajectories may contain few multi-step dependencies (retrying after a failed lookup, confirming before executing, and so on), or may include actions that rely on information the agent could not see at collection time. Of the 34,772 candidates the authors collected, 40.9% had two or fewer decisions eligible for supervised learning, and only 10.0% contained decisions made after an actual error.
Method
The method has three stages. (1) Compositional interaction synthesis: actions, observations, recovery, branching and similar elements are expressed in a shared capability vocabulary. Paths that are underrepresented in the data are selected, matching requests and initial states are generated, and a teacher model executes them in a real environment. (2) Grounded supervision: each action is checked for whether it is executable (e) and whether its arguments and tool are supported by the record visible to the agent at that point (v). Only actions that satisfy both, in tasks that succeeded and met ordering requirements, are masked as training targets. Failed calls and error messages are kept only as context. (3) Structural-support-based training: trajectories are selected to fill each path's shortfall up to a target support count per model size (1 for 1.7B, 8 for 30B-A3B, etc.). SFT is then run for 1 epoch with normalization at the parent level, so that a trajectory split into several examples still receives the same weight.
Results
- The authors report that Tau1, Tau2 and BFCL each improved across all five Qwen3 sizes from 1.7B to 30B-A3B, and the equally weighted average of the three benchmarks rose by 16.9–29.8 percentage points.
- The Tau1·Tau2 mean rose from 28.52 to 69.44 (+40.92) for Qwen3-14B and from 36.36 to 75.47 (+39.11) for Qwen3-30B-A3B.
- The trained Qwen3-4B reached a Tau2 mean of 69.17, higher than the untrained Qwen3-30B-A3B (28.33).
- The trained 8B model achieved the highest three-benchmark average among re-evaluated public agent-training checkpoints (Simia, MUA-RL, LoopTool, etc.). However, this comparison is not a controlled experiment with matched conditions.
- Gains were largest in stateful, multi-turn interactions where later decisions depend on earlier observations and feedback.
Limitations
Experiments are limited to the Qwen3 family and 1 epoch of SFT. Each task was run only once, and the authors note that comparisons with public checkpoints and closed models are not like-for-like, since training data, model families and context budgets differ. Tau1 Airline is a transductive setting that uses tasks seen during training. Synthesis and certification depend on the teacher model and environment-specific checking rules.
Why it matters
The paper challenges the common practice of scaling agent data by simply collecting more successful trajectories. It shows that when data is designed around which dependency structures it contains and whether each action can be justified by information visible to the agent, small models can match or exceed larger models in tool use. It is a useful reference when designing data pipelines for customer-support agents.
Key terms
- Decision grounding
- Whether an action used as a training target can be justified solely by the conversation and tool results visible to the agent at that point. Actions that use identifiers known only to the data collector are excluded from training.
- Compositional coverage
- Whether the training data contains enough repetitions of multi-step dependency structures in which observations determine later actions.
- Supervision mask
- A binary mask that uses the whole trajectory as context but marks with 1 only the actions on which loss is computed. Failed calls stay in the context but are excluded from training targets.
- Parent-normalized SFT
- An SFT method that gives equal weight per original interaction even when one interaction is split into several training examples. It prevents long multi-turn conversations from dominating the loss.
Original
Authors: Yihao Hu, Zhihao Wen, Yitong Lin, Siyuan Lu, Pan Wang, Wei Wu, Jiang-Ming Yang, Tao Lin · Posted 2026-10-24 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.