OpenGCUA: A Fully Open Stack for General Computer-Use Agents

Problem

Computer-use agents are shifting from screen clicking toward a mix of shell, accessibility tree, API and code execution. However, frontier systems such as GPT-6 Astra do not disclose their prompts, harnesses or execution trajectories, making it hard to study which design choices affect performance and cost.

Method

The authors defined a general computer-use agent (GCUA) as a system in which the model decides at every step 'what to observe, how to act and how to manage the task.' GUI, CLI and hybrid agents are treated as special cases in which some of these decisions are fixed in advance. The implementation has two key parts. First, GUI operations and program calls are written as a single piece of code, so conditionals, loops and verification based on intermediate results run within a single model call. Second, decisions that had been fixed by harness rules, such as when to take screenshots or how to track task state, were handed to the model (harness-to-model decision transfer). An early version of a harness for real devices and an SDK shared across computer, mobile and browser were also released.

Results

• On the OSWorld-v2 offline set, the full configuration raised the scores of all 8 models, including GPT-6 Astra and Claude Opus 5, compared with a baseline that executes GUI actions in batches (batched GUI-only). The average partial score rose by 19.6 points, and the number of steps per task fell by 53.0% on average.
• Under the same GUI and shell access conditions, code execution raised the average partial score by 6.1 points and cut steps by 14.8% compared with the GUI+CLI baseline.
• Letting the model observe the screen only when needed raised the cache hit rates of two models by 10.5 and 14.4 percentage points, respectively, without lowering scores.
• On 4 real macOS tasks, Qwen3.8-Max used 19.4 times more reasoning tokens and 5.7 times more total output tokens than GPT-6 Astra, and completed fewer tasks.

Limitations

This release does not include a trained model. The real-device harness and SDK are early versions. The real-device comparison is limited to 4 macOS tasks and selected runs, making it hard to generalize. The benchmark results are also based on a single OSWorld-v2 offline set.

Why it matters

For building business automation agents, this provides evidence that 'bundling actions into code' instead of 'clicking on screen,' and leaving the timing of observation to the model, can improve both accuracy and cost. The execution trajectories of the 8 models are released, making it easier to compare and reproduce designs.

Key terms

GCUA(Generalist Computer-Use Agent)

A computer-use agent that uses both screen operations and programmatic access such as shell and APIs, and in which the model chooses for itself how to observe, act and manage the task at every step.

Harness

The execution framework outside the model that handles providing observations, executing actions and managing context. Rules such as 'take a screenshot at every step' belong here.

Accessibility tree

Information that the operating system provides in structured form about the role, name and value of on-screen elements. It allows elements to be operated directly without coordinates.

Batched actions vs. code execution

Batched actions send several predetermined actions at once, while code execution sends a program that can branch, loop and verify based on results during execution.

Original

• arXiv abstract
• PDF
• HTML full text
• Code
• Project
• Hugging Face

Authors: Hanzhang Zhou, Long Li, Quyu Kong, Lei Wang, Panrong Tong, Long Chen, Tianyu Xia, Gongjie Zhang, et al. · Posted 2026-10-15 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.