Paper review · October 15, 2026
Agent-Reflex: Training Language Agents to Act with Reflection
Agent-Reflex: Teaching Language Agents to Act Reflectively
In environments injected with real tool errors, the authors compared successful and failed trajectories to extract reflective meta-skills, then internalized them through SFT and reinforcement learning to obtain a noise-robust tool-using agent.
Problem
In real deployments, MCP tools return timeouts or service errors, and users step in mid-run to correct the agent. Existing agent training environments, however, assume clean feedback and minimal user intervention. As a result, there is little supervision signal for teaching the reflective ability to detect and recover from failures.
Method
The method has three stages. (1) RSD: build 557 MCP-based execution environments. Group 24,510 error returns collected from real benchmarks into 8 failure types and inject them probabilistically into tool responses. Compare pairs of successful and failed trajectories from 9 different LLMs to extract 1,170 environment-specific reflective meta-skills. (2) ReSFT: use the skills to build Self-Reflection data (redoing from the error point), Skill Recovery data (restarting from the beginning) and User-Centric Reflection data (inserting user correction messages). Keep only samples that pass verification, remove the skill prompts, then run SFT. (3) ORPO: run reinforcement learning with verifiable rewards on groups mixing clean rollouts and noisy rollouts. Re-score the same tokens with and without the skill context, and add the difference as a confidence-weighted on-policy distillation signal to give dense token-level supervision.
Results
- The authors report an overall score of 58.6 across 5 long-horizon tool-use benchmarks, based on Qwen3-8B
- They report 29.1 on AgentNoiseBench, higher than environment-scaling and reflection-based baselines
- The authors report improved robustness to noise while training remained stable
Limitations
Model experiments focus on Qwen3-8B alone. The noise is injected in a form standardized into 8 failure types, so it does not represent every failure in real deployments. User corrections were produced by a simulator. The provided text was cut off partway, so per-baseline detailed numbers and ablations could not be checked.
Why it matters
Major causes of agent failure in real services are unstable tool responses and user intervention. This paper proposes a pipeline that starts from real error logs and turns recovery behavior into training data.
Key terms
- Reflective meta-skills
- Environment-specific recovery rules extracted by comparing successful and failed trajectories on the same task, in the form 'if you see this wrong pattern, fix it like this'. They can be reused on other tasks in the same environment.
- On-policy Distillation
- A method that trains the current policy, on rollouts it generated itself, to follow the token-level probabilities of a teacher with more information (here, the same model with the skill context attached).
- Noise injection
- A technique that, during training, probabilistically replaces some tool responses with error messages actually observed in practice, so the agent learns to detect and recover from failures.
Original
Authors: Guanting Dong, Xinyu Yang, Jiajie Jin, Yuyang Hu, Xiaoshuai Song, Yifei Chen, Xiaoxi Li, Jiejun Tan, et al. · Posted 2026-10-13 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.