AI Papers Reader

Personalized digests of latest AI research

View on GitHub

New Dataset and Self-Critique Technique Boost LLM Coding Skills

Researchers have unveiled OPENCODEREASONING-II, a massive new dataset designed to significantly improve the ability of Large Language Models (LLMs) to generate and critique computer code. This dataset, containing approximately 2.5 million question-solution-critique triples, is nearly twice the size of the previous largest publicly available dataset for code reasoning.

The core innovation lies in a novel “self-critique” approach that LLMs can employ during inference. This method allows the models to generate multiple potential code solutions and then use their own critique capabilities to select the best one. This “test-time scaling” technique, where models use more computational resources during inference to reason, has shown substantial performance gains, especially for smaller LLMs.

For instance, when tasked with solving a programming problem, an LLM trained with this method might generate several code snippets. It then reviews these snippets, identifying potential errors or inefficiencies in each. By analyzing its own critiques, the LLM can then pinpoint the most robust and correct solution. The paper demonstrated that this self-critique process can boost performance by up to 6 percentage points on coding benchmarks compared to simply picking the best solution at random.

The OPENCODEREASONING-II dataset was built by gathering programming questions from various competitive programming platforms. Large language models were then used to generate code solutions and, importantly, detailed step-by-step critiques of those solutions. These critiques were designed to provide justifications for the code’s correctness or flaws, along with a binary judgment (correct or incorrect).

To facilitate more comprehensive evaluation, the researchers also extended the LiveCodeBench benchmark to include support for the C++ programming language. This is particularly significant as C++ is widely used in competitive programming.

The study fine-tuned Qwen2.5-Instruct models using this new dataset through a two-stage process. The first stage focused on code generation, while the second stage jointly trained for both code generation and self-critique. The results showed that these fine-tuned models not only matched or surpassed the performance of existing state-of-the-art open-weight models in code generation but also achieved significant improvements when using the self-critique method for selecting the final output.

The research highlights that the size of the synthetic data is crucial, particularly for smaller models, leading to more substantial improvements. While larger models also benefit, the gains may plateau as they approach a performance ceiling. The study also noted promising cross-language transfer, where models trained on both Python and C++ data performed well on both languages, with a notable advantage in C++ performance.

In summary, the OPENCODEREASONING-II dataset and the self-critique test-time scaling approach represent a significant step forward in enhancing LLM capabilities for complex coding tasks, offering a more efficient and effective way to distill reasoning power into models.