AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Cracking the GeoAI Code: Researchers Peek Inside Large Language Models' Spatial Reasoning

Large Language Models (LLMs) like ChatGPT have wowed the world with their ability to generate human-like text and perform complex tasks. But how do these models “think” about geographical information? A new study, currently available as a preprint, is taking a deep dive into the inner workings of LLMs, attempting to reverse-engineer how they process and understand geospatial data.

Researchers from the University of Leicester and the University of Udine are pioneering a new framework for “geospatial mechanistic interpretability.” This involves using spatial analysis techniques to dissect the internal representations that LLMs create when processing geographical information.

“We’re essentially trying to understand ‘how LLMs think about geographic information,’” explains Stef De Sabbata, a researcher at the University of Leicester and lead author of the paper. “If such phrasing was not an undue anthropomorphism.”

To understand the approach, consider this analogy. Imagine trying to understand how a person knows that London and Birmingham are related. Instead of just asking them questions about those cities, you try to understand the network of knowledge that connects London and Birmingham. You can analyze the relationship between cities and other things they know.

In the LLM context, the researchers are looking at how LLMs respond to place names. They hypothesize that if LLMs have a true understanding of geography, their internal reactions to nearby or related places should be similar.

The researchers are leveraging a technique called “probing,” where they feed prompts containing place names into the LLM and extract the “activations” - the numerical representations of information within the model’s layers. These activations are essentially the LLM’s “thoughts” as it processes the information.

They’re then using “spatial autocorrelation” to analyze these activations. Spatial autocorrelation measures the degree to which values are clustered together in space. In this case, if activations for nearby places are more similar than activations for distant places, it suggests that the LLM has encoded some form of spatial relationship.

For instance, when the LLM is prompted with “Loughborough” and “Market Harborough” it might respond with similar activations since both are market towns near Leicester.

A key finding so far is that LLMs seem to encode geographical information in a distributed way, using many “polysemantic” neurons rather than single, dedicated ones. This makes it harder to interpret the internal representations directly. To address this, the researchers are using “sparse autoencoders” - a type of neural network designed to disentangle these complex representations into more interpretable components.

The combined use of sparse autoencoders and spatial analysis is revealing interesting patterns. For example, one “feature” extracted by the autoencoder might strongly activate for regions in southern Italy. This suggests the LLM has identified a distinct concept associated with that area.

The research is still in its early stages, but it promises to shed light on how LLMs acquire and process geographical knowledge. This could have implications for improving their performance on geospatial tasks, understanding and mitigating potential biases, and even developing more reliable AI systems for geography.