AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Small LLM Agents Achieve Giant Performance Through "Width Scaling" and Parallel Collaboration

A new paper introduces a paradigm shift in how Large Language Models (LLMs) tackle complex, broad information-seeking tasks, demonstrating that leveraging organizational capability—or “width scaling”—can outperform the traditional focus on massive individual model size (“depth scaling”).

The system, called WIDESEEK-R1, employs a hierarchical, multi-agent framework trained end-to-end using Multi-Agent Reinforcement Learning (MARL). Experiments show that WIDESEEK-R1-4B, built on a relatively small 4-billion-parameter model, achieves performance comparable to the massive 671-billion-parameter DeepSeek-R1 model on complex search benchmarks.

The Bottleneck of Depth

For years, LLM advancement has centered on depth scaling: increasing model size and employing extended, sequential reasoning like chain-of-thought to solve long-horizon problems. However, as tasks become broader—requiring the synthesis of many discrete data points into a structured format, such as filling out a large table—this depth-first approach fails.

According to the authors, single-agent methods suffer from two key limitations:

  1. Context Pollution: The agent’s memory fills up with irrelevant details from previous subtasks, degrading performance.
  2. Sequential Execution: Tasks that could be done in parallel are forced into a slow, serial chain.

WIDESEEK-R1 solves this by focusing on width scaling: breaking a broad task into independent subtasks delegated to specialized, parallel agents.

Synergizing Orchestration and Execution

WIDESEEK-R1 operates with a Lead Agent that handles task decomposition and orchestration, and multiple Subagents that execute subtasks in isolated contexts using specialized search and access tools. The key innovation lies in using MARL to jointly optimize both the lead agent’s ability to delegate and the subagents’ proficiency in parallel information gathering.

To illustrate, imagine the task is to “List the name, city, and founding year of all Ivy League universities.”

  • A traditional depth-scaled agent researches Harvard, completes that entry, then researches Yale, and so on, sequentially.
  • The WIDESEEK-R1 Lead Agent decomposes this: it simultaneously delegates parallel subtasks to individual subagents (e.g., Subagent 1 finds Harvard data, Subagent 2 finds Yale data, etc.). These subagents work in parallel, preventing context interference, and return their results rapidly for the Lead Agent to compile the final structured table.

Consistent Gains Where Deep Scaling Plateaus

The effectiveness of this approach was validated on the WideSearch benchmark, which focuses on broad information-seeking tasks requiring tabular output. The WIDESEEK-R1-4B model achieved an Item F1 score of 40.0%, matching the performance of models nearly 170 times its size.

Crucially, the researchers demonstrated the superior scaling property of width. While depth scaling (increasing the number of turns an agent can take) rapidly hits a performance plateau, WIDESEEK-R1 showed consistent performance gains as the number of parallel subagents increased, proving that the MARL framework effectively learns to coordinate growing agent swarms.

This work suggests a new path for efficient, high-performance AI, shifting the design focus from building a single, monolithic super-intelligence to architecting effective, parallel collaborative organizations, thus democratizing access to advanced reasoning capabilities.