Paper review · October 27, 2026
RSIHub: A Composable Framework for Agent Evolution Methods
RSIHub: A Composable Framework for Agent-Evolution Methods
The authors build an open-source framework that separates the policies of agent evolution methods from shared execution, evaluation and history infrastructure, and use it to implement and compare four methods under the same conditions.
Problem
Agent evolution methods use execution feedback to improve prompts, skills, tools and orchestration. However, each method rebuilds its own workspace, execution, evaluation, recovery and history management, which makes comparison hard. If a method can also modify the evaluation or records, it becomes hard to tell what caused a performance gain.
Method
The lifecycle is fixed as the stages Select, Rollout, Analyze, Mutate, Guardrail, Evaluate and Absorb, and swappable operators plug into each stage. A recipe declares the operator for each stage and the range of resources that may be changed (the mutable surface). Evaluation and the results archive are managed by the framework and kept outside the mutable surface. Each result is recorded bound to the candidate commit, its parent, a recipe snapshot and a fingerprint of the evaluation conditions. Container execution is delegated to Harbor. AHE, Hyperagents, A-Evolve and GEPA were implemented as recipes and evaluated on Terminal Bench 2 and τ³ Banking with MiniSWE and Codex as seed agents.
Results
- The largest observed overall pass@1 improvement was 13.5 percentage points.
- Method rankings changed depending on the agent, benchmark and data split, and optimization gains did not always transfer to held-out tasks.
- All four methods were implemented without modifying the framework core. Recipes were 47–61 lines, and per-method operator code was 216–1,087 lines.
- The authors also separately checked capability preservation relative to running the pre-evolution agent directly, recovery from interruptions, and enforcement of boundaries.
Limitations
The recipes port the original methods' policies at the interface level. The authors also note that differences in prompts, models and budgets mean they may not exactly match the original implementations. Evaluation is limited to two benchmarks, two seed agents and a fixed model configuration. Benchmark overfitting and specification gaming remain problems the framework does not solve.
Why it matters
As research on self-improving agents grows, fair comparison and reproducibility have become key challenges. The design of keeping evaluation outside the mutable surface and recording lineage can also inform in-house pipelines for automatically improving agents.
Key terms
- Agent evolution
- Methods that look at execution results and automatically revise an agent's prompts, skills, tools and structure.
- Recipe / Operator
- An operator is the implementation of one lifecycle stage, and a recipe is a configuration that declares which operator to use at each stage and what may be changed.
- Mutable surface
- The range of files and resources that may be modified during evolution. Evaluation code and records are kept outside this range.
- Held-out transfer
- Whether improvements hold on tasks not used for optimization. It is a criterion for gauging whether the method has overfit to the training tasks.
Original
Authors: Zimu Wang, Xiaobo Wang, Siyu Ye, Jiaqi Li, Silin Chen, Yuling Shi, Ruobing Wang, Jingjing Zhang, et al. · Posted 2026-10-23 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.