AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Large Language Models Meet Code Repositories: CODEXGRAPH Enables Precise Code Navigation and Understanding

Large Language Models (LLMs) have shown impressive capabilities in code-related tasks. However, they struggle when dealing with entire code repositories, where complex dependencies and intricate project structures pose significant challenges. Current solutions for enhancing LLM-codebase interaction at a repository scale typically rely on similarity-based retrieval or manual tools and APIs, each with notable drawbacks. Similarity-based retrieval often has low recall in complex tasks, while manual tools and APIs are typically task-specific and require expert knowledge, reducing their generalizability across diverse code tasks and real-world applications.

To address these limitations, researchers from the National University of Singapore and Alibaba have introduced CODEXGRAPH, a system that integrates LLM agents with graph database interfaces extracted from code repositories. By leveraging the structural properties of graph databases and the flexibility of the graph query language, CODEXGRAPH enables the LLM agent to construct and execute queries, allowing for precise, code structure-aware context retrieval and code navigation.

Imagine a software engineer working on a large codebase and wanting to quickly understand the relationship between different classes and methods. With CODEXGRAPH, the engineer can ask a natural language question like “Which classes inherit from the TaskManager class and what methods do they have?” CODEXGRAPH will automatically translate this question into a graph query, navigate the code repository’s graph database, and return the relevant information – a list of classes inheriting from TaskManager and their corresponding methods – without requiring manual code exploration or complex code search.

CODEXGRAPH leverages a unified graph database schema that defines nodes representing source code symbols and edges representing relationships between those symbols. This schema allows for comprehensive analysis of code structures, including multi-hop reasoning and global analysis. When a user provides a code-related input, the LLM agent analyzes the required information from the code graphs, constructs flexible queries using graph query language, and locates relevant nodes or edges. This enables precise and efficient retrieval, allowing for effective scaling to larger repository tasks.

The researchers evaluated CODEXGRAPH on three challenging repository-level code benchmarks: CrossCodeEval, SWE-bench, and EvoCodeBench. Their results demonstrate that CODEXGRAPH achieves competitive performance across all benchmarks, especially when equipped with more advanced LLMs. Additionally, they developed five real-world coding applications, including code debugging, code commenting, and code generation, showcasing CODEXGRAPH’s versatility and efficacy in practical software engineering tasks.

CODEXGRAPH represents a significant step forward in bridging the gap between LLMs and code repositories. By leveraging the power of graph databases and LLMs, CODEXGRAPH offers a novel and efficient approach to code navigation and understanding. This system has the potential to revolutionize how software engineers interact with large codebases, enhancing productivity and fostering innovation.

The research paper, “CODEXGRAPH: Bridging Large Language Models and Code Repositories via Code Graph Databases,” is available on arXiv. It provides a detailed explanation of the system’s architecture, its evaluation results, and its potential applications in real-world software engineering.