AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Less is More: Why Simple AI Beats 'State-of-the-Art' When Data is Scarce

For the past decade, deep reinforcement learning (RL) has driven some of artificial intelligence’s most headline-grabbing achievements, from mastering complex video games to controlling robotic systems. But a quiet crisis has been brewing in the laboratories where these algorithms are designed. According to a provocative new paper by researcher Ezgi Korkmaz, a fundamental, unexamined assumption has steered a major branch of AI research down a misleading path.

The assumption is simple: if an AI algorithm reigns supreme when trained on mountains of data, it must also be the best choice when data is scarce. However, Korkmaz’s research reveals that the relationship between data volume and algorithmic performance is “non-monotonic”—meaning what works best with unlimited practice can fail spectacularly when training time is short.

To build an intuition for this, imagine preparing two people for a complex strategy game. Student A is given a massive, 1,000-page encyclopedia detailing every possible move, while Student B is given a simple, one-page cheat sheet of basic rules. If both students have ten years to study (a high-data regime), Student A will inevitably become a grandmaster. But if they only have ten minutes to prepare (a low-data regime), Student A will get bogged down in the opening chapters and freeze, while Student B will easily grasp the basics and play a decent game.

In AI, the “encyclopedia” represents high-capacity models like Distributional RL (such as C51, QRDQN, or IQN), which try to learn the entire probability distribution of future rewards. The “cheat sheet” represents simpler, lower-capacity architectures like the classic “Dueling” network, which merely estimates average outcomes.

Through rigorous mathematical proofs and extensive testing on the classic Arcade Learning Environment (ALE), Korkmaz demonstrated that high-capacity models suffer from high “sample complexity.” They require exponentially more experience to learn stable behaviors. When restricted to the popular “100K benchmark”—where an AI is allowed only 100,000 frames of gameplay (roughly two hours of human play)—these advanced models struggle to make sense of the noise.

Alarmingly, Korkmaz found that a basic “Dueling” algorithm from 2016 routinely outperformed newer, highly praised “state-of-the-art” algorithms in the low-data regime. Because researchers implicitly assumed that the high-data champions would naturally scale down, they often omitted this simple 2016 baseline from their comparative benchmarks. This blind spot led to years of papers claiming breakthrough performance gains that were, in reality, statistically inflated.

“Our results demonstrate that a line of reinforcement learning research under the canonical design and evaluation paradigms resulted in incorrect conclusions,” Korkmaz writes.

The paper calls for a “principled evaluation framework” in AI design, urging researchers to abandon the lazy assumption of monotonicity. For industries where gathering data is slow and expensive—such as healthcare or autonomous driving—this shift is vital. By matching the complexity of an AI’s architecture to the actual volume of data available, engineers can avoid over-engineered failures and build systems that are genuinely efficient from day one.