AI Papers Reader

Personalized digests of latest AI research

View on GitHub

To teach browser agents the web, Microsoft builds synthetic worlds that repair themselves

Computer-use agents—AI models that operate a browser by clicking, typing, and reading screens—learn from consequences. But the applications where their work matters most, email, banking, health records, sit behind logins, write to real accounts, and cannot be reset. Microsoft Research’s Echoverse generates synthetic applications whose databases researchers own. State changes are real but safe to break and quick to reset, and a task is graded by comparing database state before and after, not by judging screenshots.

The central idea is co-evolution: every graded rollout is read twice. Failures traced to the app, its tasks, or its grader become repairs to the world; failures on valid, verified tasks become training signal for the model. The world is repaired before model failures are trusted, and researchers say goals are never weakened just to raise pass rates.

A concrete example shows why this matters. In ECHOSTAY, a lodging-booking world, an agent-driven guest-count control silently failed, so correct bookings could never register. Repairing it raised the share of completable bookings from 48% to 78%, restoring 15 of 24 blocked tasks, and the model trained on the repaired world more than doubled, from 16.2% to 38.5% task success.

The headline supervised result: a 9-billion-parameter Qwen3.5-9B model trained on 21,009 verified trajectories from twelve worlds improved from 36.5% to 67.1% success across fourteen evaluation splits, a gain of 30.6 percentage points and roughly a doubling. That left it within 13.6 percentage points of GPT-5.4, the far larger frontier model whose successful trajectories produced the training data. On the six weakest baseline splits, gains were three- to nine-fold.

Depth, not just volume, appears to drive these gains. In a small controlled comparison on two live WebVoyager domains, a model trained on shallow worlds fell below the base model on Allrecipes, 80.0% to 75.0%, and stayed flat on Hugging Face at 48.0%. Training on deep worlds, where early actions constrain later state, raised both to 85.0% and 65.0%. The authors caution this is two domains and a few dozen tasks, but it suggests a shallow clone can be worse than no training at all.

Scaling experiments point the same direction. Adding trajectories within fixed environments eventually stopped helping live-web transfer (WebVoyager moved only from 54.8% to 55.6% between 6,400 and 20,000 trajectories), while adding new environments continued to improve scores. That is evidence for environment diversity over raw volume.

The same worlds also work as reinforcement-learning environments. Starting from the supervised policy, a run over five worlds with a composite reward—the grounded database verifier plus a dense per-step judge—raised held-out judged success from 58.8% to 68.0%. The tradeoff: the per-step term relies on a vision model judging screenshots, reintroducing the subjective signal the grounded verifier was designed to avoid. The researchers state that validation scores use the grounded outcome alone.

The broader significance is not that synthetic worlds replace the web. It is that training data can be treated as something to maintain, repair, and improve in response to the agent’s own failures, rather than as a fixed collection. The results establish a compounding loop, not a solved problem.