AI Agents Play Test Their Own Games to Make Them More Rewarding
Researchers have built an automated system that uses AI game-playing agents to revise generated games so that players improve with practice, a quality the authors call mastery. The problem matters because AI tools can now produce playable games from text instructions, but a game that runs is not necessarily one that holds people’s interest. Designing for mastery, where early results are modest and skilled play is clearly rewarded, has usually required extensive human testing and tuning.
The system, called PlayCraft, alternates between two stages. In the playing stage, a “player agent,” a foundation model that reads rendered game frames and presses controls, pursues the game’s objective over ten episodes while retaining its memory. After each episode, it writes a report on its progress, decisions, and difficulties. In the crafting stage, a “crafter agent” reads those reports, estimates the player’s baseline abilities, and predicts how beginner, intermediate, and expert players would perform. It compares these predictions with target anchors, under which beginners should reach 10–30% and intermediates 50–70% of expert performance, and then revises the game code to close the gaps. The authors ran three such cycles per game.
The paper’s Breakout example shows the mechanism. In the first revision, bricks gained armor that retained damage from underside hits, so one shot could prepare a brick for a later one. Player reports then described shots that struck the wrong column, prompting a second revision that required three underside hits per brick. A third revision made the rebound angle less sensitive to small changes in impact position, letting the ball pass through a brick on the third hit.
To test the approach, the researchers used ten browser-based games: five 2D recreations, such as Flappy Bird and Snake, and five 3D games generated from prompts. They compared the results against two baselines. “Code” revised games using only a coding agent’s analysis of the source, while “Policy” used controllers that act on internal game state and do not retain memory. Player strength was varied across three settings, Low, Medium, and High, defined by model and reasoning effort.
The main metric, learning gain, compares normalized scores from early episodes with those from late episodes and is reported in percentage points. PlayCraft’s macro-average gain was 13.7 at Low, 31.5 at Medium, and 34.3 at High. Relative to the initial games, these are increases of 2.7, 14.0, and 21.9 percentage points. Learning slope, the fitted trend across episodes, was 4.61 at Medium and 4.74 at High, up 2.09 and 2.64 from the initial games. At Low, PlayCraft’s slope of 1.95 was essentially tied with the initial games’ 1.96. PlayCraft beat both baselines on both macro-averages at every setting.
The results also reveal weaknesses. In Robotic Arm, gain and slope dropped sharply after the first revision, which added wrist sway and varied can positions. The authors suggest these demands made learned routines harder to repeat, and later revisions only partly recovered. In Drone Delivery at Low, players usually failed to reach the second delivery because of battery exhaustion or collisions, and PlayCraft trailed the initial game on both metrics. The study also rests on an unverified assumption that agent play resembles human play, and longer play budgets raise time and inference costs. The authors’ explanation for the smaller gains from Medium to High is that the Medium configuration generated the records used for revision, though this is a hypothesis.
The work suggests that detailed, memory-equipped agent playtesting can guide designs toward skill-based progression. It does not yet show that the revised games are more enjoyable to people. Comparing agent and human learning patterns remains the key next step.
Chat about this paper
To chat about this paper, you'll need a free Gemini API key from Google AI Studio.
Your API key will be stored securely in your browser's local storage.