Paper review · November 12, 2026
World In-Context Learning: In-context learning for agentic robot manipulation
World In-Context Learning for Agentic Robot Manipulation
By having a frozen model rehearse tasks in a simulator and record the feedback in its context rather than its weights, the method achieved higher manipulation success rates than policies trained on a single demonstration.
Problem
Trained robot action policies store skills in their weights, which ties them to the training data distribution. In-context learning avoids this problem, but its performance has fallen short of trained policies. Demonstrations recorded in other scenes contain no feedback from the environment the robot currently faces.
Method
The authors argue that what an agent learns in context is semantic information, such as task steps and object roles, rather than visual appearance, so it can transfer to other environments. Building on this, they propose WICL (World In-Context Learning). A frozen model starts from a single demonstration, rehearses the task in a simulator and writes the feedback into its context. The RoboScript infrastructure that implements this freezes the learned knowledge as Task Memory and deploys it to new scenes. At deployment, data is isolated so that only inputs available on a real robot are used.
Results
- Across all 50 tasks in RoboTwin 2.0, with one demonstration per task and no gradient updates, it reached an average success rate of 83.4% over the clean and randomized settings.
- The authors report that this is more than 20 percentage points higher than the strongest trained action policy.
- The authors report that it generalizes under visual distribution shifts, improves with reflection and scales with the performance of the foundation model. Through cross-embodiment memory adaptation, it was also transferred to a real dual-arm robot.
Limitations
This analysis is based on the abstract. Without access to the full text, details such as the scale and success rates of the real-robot experiments, the cost of simulator rehearsal and the degree of dependence on the foundation model could not be verified. The main figures are results on a simulation benchmark (RoboTwin 2.0).
Why it matters
If robots can learn new tasks from context alone without retraining, the cost of collecting data and training for each task drops sharply. Using a simulator as a "practice space" is a useful reference for designing robot agents built on foundation models.
Key terms
- In-Context Learning(ICL)
- A way of performing a new task using only examples or feedback placed in the input context, without changing the model's weights.
- Task Memory
- A fixed context record of task knowledge gained through simulator rehearsal. It is carried over as-is to new scenes.
- Cross-embodiment
- Transferring knowledge gained on one robot (or simulator) to a robot with a different physical form.
Original
Authors: Heng Fang, Zehui Guo, Yuehan Zhang, Mingyang Du, Dingkang Liang, Xiang Bai · Posted 2026-11-09 · License arXiv 비독점 배포 라이선스 · Based on the abstract
This review is an AI-written summary of the paper. Please treat the original paper as the reference.