AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Same Tools, Different Behavior: How Interface Format Shapes AI Agent Decisions

Language agents, which are AI models that act by calling tools or issuing commands in software environments, increasingly reach those capabilities through one of two interfaces. The Model Context Protocol (MCP) presents each capability as a named tool with structured arguments. A command-line interface (CLI) expresses the same operations as typed commands and flags. Developers often treat the choice as plumbing. A new study from researchers at the Hong Kong Polytechnic University and the Chinese Academy of Sciences suggests that the choice may change what a model decides to do.

Isolating that effect is difficult. Comparisons of MCP and CLI systems usually change several things at once, including prompts, tool granularity and validation. To address this, the authors built a framework called Matched Interface Comparison and Analysis (MICA). It generates MCP and CLI interfaces from a single semantic action contract, meaning a shared specification of the operations and arguments available, and sends each decoded action through the same validation and execution runtime. Only the model-facing presentation differs.

Across six models and three environments (ALFWorld for household tasks, WebShop for online shopping, and ScienceWorld for scientific reasoning), the interface mattered substantially. For Qwen3-14B on ALFWorld, the score was 56.7% under CLI and 20.1% under MCP, a gap of 36.6 percentage points. On WebShop, the same model scored 64.4% under MCP and 59.5% under CLI. Token costs also reversed direction: MCP used 54.9k more tokens per ALFWorld task but 35.0k fewer on WebShop.

The authors trace some of these gaps to recovery after errors. On WebShop with Qwen3-4B, retry acceptance after a failed action was 50% under CLI and 77% under MCP, and later task success was 18% and 34%, respectively. The authors argue that these different recovery paths alter the interaction history that conditions later decisions. Across 2,004 paired episodes, 1,889 (94.3%) showed a structured divergence in the models’ choices, and the authors report that these divergences typically appeared early and then compounded.

The researchers also propose DART (Decoupled Action Realization and Targeted Repair). Once a semantic action is validated, DART keeps it fixed. If its interface-specific expression fails, the system repairs only that expression rather than choosing a new action. Tested on Gemini-2.5-Flash against a matched baseline, DART raised scores in all six benchmark-interface settings. The gains were 1.5 points (CLI) and 9.0 points (MCP) on ALFWorld, 0.8 and 4.4 points on WebShop, and 6.7 and 5.3 points on ScienceWorld. Under MCP, token use fell by 27.1k, 11.2k and 4.3k per task across the three environments. Under CLI, token changes were mixed.

The study has clear limits. DART was evaluated on a single model, and the matched contracts were built by the authors for these benchmarks, so real-world MCP and CLI servers may differ. The reversals in Table 1 show that no interface wins consistently, and the explanations for the differences are the authors’ interpretations rather than established causes.

Still, the work supports a practical point: interface design can shape how an agent behaves, not just what it can access. It does not show that either protocol is superior in general.