AI Models Act Like Mathematicians, Constantly Rewriting Their "Code" for Simplicity
In a new study exploring how deep neural networks organize knowledge, researchers from Google DeepMind, Brown University, and Université de Toulouse have introduced a novel metric to quantify feature complexity, revealing that models actively simplify their most crucial learned concepts over time, a process likened to a “sedimentation” of foundational elements.
The findings provide crucial context for the phenomenon of “simplicity bias,” where AI models often rely on computationally simpler input features, sometimes leading to undesirable “shortcut learning.”
To measure complexity, the team leveraged V-information, a concept from information theory that assesses how many computational transformations a feature requires to be extracted. Using this metric, they analyzed over 10,000 features extracted from a standard ImageNet-trained ResNet50 vision model, mapping them across a spectrum of simple to complex.
Simple features—those requiring minimal computation—include basic visual elements like color detectors (e.g., “sky,” “grass”) and low-frequency patterns (“bokeh,” “lines”). In contrast, complex features require intricate, sequential processing, encompassing structured object components like “insect legs,” “whiskers,” and “ears.”
The analysis showed that these complex features not only emerge later during the training process than simple features, but they also rely heavily on the model’s primary computational pathway (the “Main” branch). Simple features, once computed in early layers, are primarily “teleported” via residual connections—architectural shortcuts inherent to ResNet—ensuring they are instantly accessible throughout the rest of the network without further transformation.
The researchers then investigated the critical relationship between a feature’s complexity and its importance in driving the model’s final decision. They confirmed a clear simplicity bias: simple features exert a disproportionate influence on the network’s predictions.
However, the most surprising revelation came from tracking the training dynamics. While the network initially learns features of increasing overall complexity, a phase soon begins where the model actively reduces the complexity of its most important features.
This behavior suggests the ResNet model acts akin to a Levin Machine—a theoretical concept of an optimal search algorithm—that continuously seeks the shortest, most computationally efficient “program” to achieve its goal. In practice, this means the most vital features are compressed and pushed to earlier layers of the network, ensuring they are accessible via a shorter computational graph.
This “sedimentation process,” where foundational and predictive features become computationally cheaper over time, suggests neural networks are intrinsically driven towards computational efficiency and generalization, providing a vital new angle for researchers aiming to curb shortcut learning and build more robust, reliable AI systems.
Chat about this paper
To chat about this paper, you'll need a free Gemini API key from Google AI Studio.
Your API key will be stored securely in your browser's local storage.