Paper review · October 13, 2026
SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthetic Software Environments
SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments
A multi-agent pipeline automatically synthesizes 182 GUI desktop apps, and training on their Trajectories raised OSWorld Pass@3 from 41.39% to 56.97%.
Problem
Training computer-use agents (CUAs) requires diverse, reliable execution environments. Real native apps are costly to integrate, which narrows app coverage and risks overfitting to specific apps. Browser-based synthetic environments lack interaction depth and workflow complexity.
Method
It is an automated pipeline with three stages: design, implementation and evaluation. In the design stage, minimal input is expanded into a detailed functional specification, and several structurally different Avalonia UI blueprints and SVG icons are created. Layout-tree edit distance is used to enforce differences between variants. In the implementation stage, a coding agent builds the app and verifies it at the source level. In the evaluation stage, a GUI agent runs checkpoint-based tests, and the app is revised iteratively from failure logs, up to three versions. Tasks at different difficulty levels are synthesized only from GUI-verified features. The collected Trajectories were used for multimodal SFT of Qwen3.5-9B.
Results
- It synthesized 182 runnable apps across 12 categories, with an average GUI task pass rate of 91.5%.
- A model trained only on SoftwareGym Trajectories, without using any OSWorld apps or tasks, raised OSWorld (244 tasks) Pass@3 from 41.39% to 56.97%, a gain of 15.58 percentage points.
- It scored 40.0% Pass@3 on SoftwareGym-Bench, made up of 120 tasks across 12 categories. The authors report this is higher than some larger specialized CUAs.
- The authors report that it takes far less time and cost than manual development.
Limitations
The synthetic apps are limited to Avalonia-based desktop apps. It uses commercial models: GPT-5.5 for design and implementation, and Seed 2.1 Pro for GUI evaluation. Training used SFT only. The trained model's OSWorld score (56.97%) is still below API models such as GPT-5.5 (71.31%). Evaluation is Pass@3 with temperature 1 over 3 Rollouts.
Why it matters
It shows that diverse GUI training environments can be created at scale without integrating real apps one by one, and that the data transfers to unseen real apps. This is one way to ease the data bottleneck for computer-use agents.
Key terms
- Computer-use agent (CUA)
- An AI agent that looks at the screen and performs real software tasks by operating the mouse and keyboard.
- OSWorld
- A benchmark that evaluates how well agents complete tasks in real desktop apps such as Chrome, GIMP and LibreOffice.
- Pass@3
- A metric that counts a task as solved if at least one of three independent attempts succeeds.
Original
Authors: Tian Jin, Wenhao Wang, Jing Kang, Haoting Shi, Rui Ye, Yijun Lu, Siheng Chen · Posted 2026-10-08 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.