Paper review · October 14, 2026
Scaling Agentic Data for Long-Horizon Terminal Agents
Scaling Agentic Data for Long-Horizon Terminal Intelligence
The authors built an engine that automatically synthesizes long-horizon terminal task environments from real-world problems, then fine-tuned Qwen3.5-27B on about 3,000 trajectories, improving performance on all 12 benchmarks.
Problem
Existing methods for synthesizing terminal environments produce short trajectories. This makes it hard to provide experience with long-horizon work where exploration, strategy revision and verification span many steps. In addition, the core work is often pre-completed or simplified while the environment is being built, so the agent skips the exploration and implementation it should be doing.
Method
TerminalHorizon has two stages. ① Challenge-Guided Long-Horizon Synthesis: Planner, Deep Research and Construction agents create tasks based on real-world materials. An independent agent not involved in creation attempts each task, finds missing or pre-completed work, and the environment is rebuilt. ② Progressive Horizon Expansion: tasks are gradually extended by adding new requirements under the same goal that depend on earlier results. This produced 1,535 environments, and 2,852 trajectories were collected using DeepSeek-V4-Pro as the teacher.
Results
- The average trajectory length was 52.8 turns, with a 90th percentile of 98 turns. These were longer than other synthesis methods compared under the same teacher.
- TerminalHorizon-27B scored 14.2% higher on Terminal-Bench 2.0 and 13.85% higher on 2.1 than the base model Qwen3.5-27B.
- AutomationBench Pass Rate improved by 11.7% and Partial score by 23.3%, and VitaBench improved by 11.3%.
- In a recursive self-improvement (RSI) case study with a 12-hour research budget, the authors reported that the model kept experimenting after its first solution and improved its approach based on feedback.
Limitations
Only SFT was used, with a single base model (Qwen3.5-27B). Data quality depends on a specific teacher model (DeepSeek-V4-Pro). The RSI result is a case study, not a quantitative comparison. Within the provided text, it is hard to confirm whether the reported improvements are absolute score differences or relative percentages.
Why it matters
This shows that in long-horizon agent training, whether tasks are designed to require real exploration, implementation and verification matters more than the amount of data. The fact that about 3,000 trajectories were enough to have an effect is meaningful in terms of data-building cost.
Key terms
- Long-horizon Trajectory
- A record of an agent interacting with an environment over dozens of turns or more to carry a task through to completion.
- Independent Challenger
- An agent not involved in creating the task. It attempts the task itself to check whether the intended work is actually required.
- Terminal-Bench
- A benchmark that evaluates an agent's ability to carry out real-world tasks in a terminal environment.
Original
Authors: Shun Zou, Ziyu Ma, Yi Zou, Yong Wang, Lin Chen, Zehui Chen, Guanhua Chen, Xiangxiang Chu, et al. · Posted 2026-10-09 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.