Paper review · October 10, 2026
AgentGarten: a code-based virtual world that helps agents keep improving
AgentGarten: Code Worlds for Evolving Agents
The authors combined simulators and game engines with a shared real-time neural renderer to build interactive environments written in code, and showed agents learning within them in just a few rounds.
Problem
What an agent can learn is bounded by the environment it practices in. Simulators and game engines have clear states and rules, but making them visually diverse requires costly 3D asset production. Video world models, by contrast, are visually rich, but their states and physics rules are implicit inside the model, which makes them hard to inspect or modify.
Method
The engine executes scene state and interaction rules as code and outputs structured conditions such as depth and surface normals. The shared neural renderer takes these conditions, appearance reference images, text and past frames, and generates the observed frames. The renderer starts from Cosmos 3-Nano, is trained on geometric conditions, is then converted to block-wise autoregressive generation, and is Distilled with Adversarial Forcing, which the authors propose. Adversarial Forcing combines DMD (Distribution Matching Distillation) on the model's own Rollouts, bit-exact recomputation (exact replay) that lets gradients reach past block encodings, and adversarial training on real video with exact R1/R2 regularization. Agents act only on the rendered frames, and after each round they summarize their experience in a playbook passed on to the next agent.
Results
- The authors reported that agents learned within 4 rounds, far more efficient than conventional reinforcement learning, which needs millions of attempts.
- The authors reported that in a hide-and-seek environment, strategies such as using ramps and building shelters appeared within a few games.
- The authors explain that exact replay achieves recomputation bit-identical to the sampling Rollout, eliminating the numerical mismatch of the SGF approach, which computes the entire sequence at once.
- The authors state that the renderer is optimized for real-time interaction on a single GPU.
Limitations
Because the provided text was cut off partway, the renderer quality comparison (vs. Self Forcing) and quantitative results for each environment could not be checked. The comparison with reinforcement learning is given only at the level of '4 rounds vs. millions', so the different comparison conditions (a pretrained foundation-model agent vs. RL trained from scratch) should be taken into account.
Why it matters
Splitting state and rules into code and frames into a shared generative model makes it possible to add new environments with code alone, without assets. This could be a way to create environments for agent training at scale.
Key terms
- Code World
- An environment where a scene program executes state and rules and a neural renderer generates the frames. Rendering errors do not accumulate in the state.
- Adversarial Forcing
- A Distillation technique that trains on the model's own Rollouts. It combines distribution matching, exact replay that passes gradients through to past encodings, and adversarial training on real video.
- DMD(Distribution Matching Distillation)
- A Distillation method that trains a Student model so that the distribution of its generated samples matches the Teacher model's distribution, allowing generation in fewer steps.
- Playbook
- A document in which the agent writes down, after each round, the causes of failure and what it learned. The agent in the next round takes it over and improves on it.
Original
Authors: Jiawei Chi, Shangchen Miao, Zhiyuan Shi, Kailu Wu, Hanyang Wang, Weiliang Chen, Qiyu Dai, Jinshan Ren, et al. · Posted 2026-10-08 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.