AI Papers Reader

Personalized digests of latest AI research

View on GitHub

AI Builds a Better Mobile Agent Through a Closed Loop of Data, Training, and Runtime Support

Mobile agents must turn high-level requests into reliable, multi-step actions across apps, but real-device interaction is costly and hard to parallelize. Researchers at Alibaba Token Hub’s MAI Team built Qwen-Planner-Agent to test a broader idea: can AI help improve AI by converting execution feedback into coordinated changes in training data, model learning, and runtime support?

The team’s closed-loop AI-for-AI framework connects three stages through a shared action-feedback-verification contract. AI for Data uses specialized agents to construct executable tasks, collect interaction trajectories, curate and balance training data, and feed training results into later data generation. AI for Training combines a supervised planning cold start with hybrid-environment online agentic reinforcement learning across programmatic sandboxes, LLM-simulated environments, and selected real-device sessions. CARE—Competence-Aware Reward-and-Advantage Engineering—adapts reward schedules and calibrates advantages as task competence changes, aiming to cut reasoning and tool-use costs while retaining task success. AI for Harness places a runtime layer between the planner model and deployment environment. It manages skills, persistent memory, tools, and structured execution feedback; failure traces can guide reviewed, versioned updates to the model and harness.

A qualitative trace shows the runtime recovery mechanism. Qwen-Planner-Agent tried to convert 70 kilograms to pounds. The conversion tool rejected both full unit names and abbreviations. The agent instead used a calculator tool with the standard conversion factor to obtain 154.32 pounds, then sent that result to the coach identified in the interaction history, preserving a data dependency.

On MobilePA-Bench, with 1,700-plus executable tasks, 200-plus mobile tools, and 13 query-task domains, Qwen-Planner-Agent 27B achieved an Overall score of 77.05%, the highest among evaluated models and systems. It exceeded GPT 6 Astra (76.84%) and Claude Opus 5 (75.71%). Both planner sizes improved over Qwen baselines: 27B rose from 67.22% to 77.05%, while 35B-A3B rose from 54.90% to 69.91%. The 27B agent also scored 77.79% on Tool Use, 74.76% on Memory, 86.25% on Skills, and 59.55% on Sub-agent coordination. With the 27B checkpoint fixed, adding the harness raised Overall from 71.90% to 77.05%.

Training experiments reported that CARE maintained average accuracy comparable to Vanilla RL while reducing output length by 32.5% at the final plotted step. In long-history memory tests, matching Qwen model pairs improved mean scores with the harness from 42.18% to 68.29% on BEAM-500K, 40.41% to 70.27% on BEAM-1M, and 23.42% to 61.66% on BEAM-10M. On MobilePA-Internal, alternating model training and harness refinement improved overall performance from 82.67% to 88.50% after four iterations; on MCPMark it rose from 38.00% to 46.98% after three—gains of 5.83 and 8.98 percentage points over the model-only baseline.

The paper’s cost comparison estimates Qwen-Planner-Agent’s output cost at $2.41 per 1,000 tasks, versus $3.06 to $67.76 for other evaluated models. But those estimates exclude input tokens, external tool charges, device execution, and additional harness processing. Model-harness co-evolution remains a reviewed development pathway, not established autonomous co-evolution. The work suggests AI-assisted feedback loops can improve mobile planning and some general agentic tasks, but within evaluated settings and with human oversight still in the loop.