Recursive Game Creator: A Play-Experience-Centered Harness for Agent-Based Game Development

Problem

Coding agents can quickly build games that run. But a program working correctly does not guarantee the game is fun. Existing methods had GUI agents test games by looking at screenshots. This is slow, misses game state information and makes it hard to collect varied play data. As a result, there were limits to iteratively improving games based on 'play experience'.

Method

The authors build an iterative loop of four roles. The Designer makes a plan (improvement hypotheses, changes, acceptance criteria, required assets) based on user instructions and review feedback. The Builder drafts, generates assets and integrates them into a candidate game. The Coding-Native Player uses the game's programming interface to write policy code with different strategies, skill levels and exploration tendencies. These policies are run repeatedly and in parallel to collect Trajectories. The Reviewer combines common metrics such as success rate and play time, genre-specific rubrics, visual evidence such as screenshots, and user preferences. It then compares the candidate and the previous version A/B with version information hidden, keeps the better one, and produces feedback for the next round.

Results

• On 45 selected tasks from GameCraft-Bench, three rounds of improvement raised the overall score from 72.70 to 77.89, a gain of 5.19. Scores improved in every reported category and were the highest among the compared methods.
• The task success rate under strict criteria on GameASG-Bench was 53.2% (25/47). This is 34.1 percentage points higher than a baseline using the same model.
• The average runtime-check pass rate on GameASG-Bench was 93.4%, the highest among the compared methods.
• The authors report that in a user study, games made with this method had longer play times and higher ratings.

Limitations

As the authors acknowledge, policy-based players are only proxy strategies and do not represent real human player populations. Proxy metrics such as play time alone cannot prove that a game is fun. The method applies only when the game has a programming interface that exposes state and actions. Only a subset of GameCraft-Bench tasks (45) was evaluated. A full cost analysis covering policy generation, revision, execution and review is hard to find in this excerpt. The code has not been released yet.

Why it matters

This is an attempt to shift the goal of agent-based generation from 'output that works' to 'quality of user experience'. Collecting play data quickly with code policies, instead of GUI testing that calls a model for every action, could also be applied to evaluating interactive software beyond games.

Key terms

Harness

An execution framework that ties together multiple agents, tools and evaluation steps to run an iterative workflow.

Coding-Native Player

A testing agent that, instead of looking at the screen and deciding each time, writes and runs policy code that reads the game state and chooses actions, collecting play records.

Trajectory

A sequential record of the states, actions, events, progress, time and end result logged during one play session.

Game-specific rubric

A scoring sheet that sets separate evaluation criteria that matter for each genre. For example, a racing game is judged on steering responsiveness, and a visual novel on dialogue consistency.

Original

• arXiv abstract
• PDF
• HTML full text
• Code
• Project
• Hugging Face

Authors: Jiajun Chen, Haoyu Wu, Mingda Jia, Xihui Liu · Posted 2026-10-06 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.