AI Papers Reader

Personalized digests of latest AI research

View on GitHub

ServiceNow's DeepCodeSeek Revolutionizes Enterprise Code Generation with Real-Time API Retrieval

San Francisco, CA – Developers working within the complex ServiceNow ecosystem often struggle to find the right pre-built code components, known as Script Includes, to complete their tasks. Traditional search methods fall short, failing to understand the nuanced intent behind incomplete code snippets. To address this critical gap, ServiceNow researchers have developed DeepCodeSeek, a sophisticated, multi-stage API retrieval pipeline designed to deliver highly relevant Script Includes in real-time, significantly boosting code generation quality and efficiency.

At its core, DeepCodeSeek tackles the challenge of accurately retrieving relevant Script Includes from partial code without explicit user queries. Imagine a developer is writing a ServiceNow script to manage user permissions. They start typing var user = gs.getUserID(); and then need a function to check if that user belongs to a specific group. Without knowing the exact name of the available Script Include, a simple keyword search might return hundreds of unrelated results. DeepCodeSeek aims to predict the exact Script Include needed at that precise moment.

The system employs a novel multi-stage approach. First, it leverages a Knowledge Graph built from ServiceNow platform metadata to dramatically reduce the search space, filtering down the millions of potential Script Includes to a manageable subset. This is akin to knowing that the user permission task likely involves “security” or “access control” modules within ServiceNow, immediately narrowing the search.

Next, DeepCodeSeek utilizes an enriched indexing strategy. Instead of just indexing raw code, it structures Script Includes by grouping methods under their parent namespaces and enriches them with JSDoc documentation and API usage examples. This structured approach, akin to having a well-organized library with detailed summaries and examples for each book, allows the system to better distinguish between similar functionalities and reduce ambiguity. For instance, instead of just searching for “user” and “group”, the enriched index can understand the specific relationship between a UserUtil Script Include and its isInGroup method.

To further refine the search, DeepCodeSeek incorporates LLM-Powered Code Expansion. When a developer provides only a partial code snippet, a Large Language Model (LLM) intelligently infers the developer’s intent and generates a more descriptive query. This is like having a smart assistant who, upon seeing var user = gs.getUserID();, might proactively suggest searching for functions related to “user group membership” rather than just “user”.

The pipeline culminates in a reranking stage. Even with the best initial retrieval, the most relevant Script Include might not be at the very top of the results. A sophisticated reranking mechanism, optimized through a comprehensive post-training pipeline involving synthetic dataset generation, supervised fine-tuning, and reinforcement learning, ensures that the top candidates are precisely the ones the developer needs. This process is so effective that a compact 0.6 billion parameter reranker model developed by ServiceNow now outperforms a much larger 8 billion parameter model, achieving 87.86% top-40 retrieval accuracy while operating 2.5 times faster. This optimization is crucial for delivering real-time code completion without lag.

The impact of DeepCodeSeek is significant, promising to accelerate development cycles within ServiceNow environments by providing developers with instant access to the precise, contextually relevant code components they need, directly within their workflow.