AI Papers Reader

Personalized digests of latest AI research

View on GitHub

MindForge: How AI Agents Are Learning to Build Software From Scratch

Artificial intelligence has made impressive strides in assisting software developers, but today’s AI coding agents have a hidden weakness: they are mostly built to patch existing code. Ask an AI to fix a bug in a GitHub repository, and it often succeeds. Ask it to build a complex software program entirely from scratch, and even top-tier models fail over 99% of the time.

To bridge this gap, researchers have developed MindForge, an automated framework that trains smaller, open-weight language models to master the entire software engineering life cycle—from initial design to final testing.

Rebuilding Software Without the Blueprint

Most coding datasets train AI agents on visible, human-written source code. MindForge takes a radically different “source-free” approach.

Imagine giving a builder a finished house and a user manual, but hiding the architectural blueprints. The builder must inspect how the house functions, infer how it was constructed, write new plans, build it from scratch, and test every door until the replica matches the original.

MindForge automates this process for software. It turns open-source command-line tools into cleanroom training environments. The AI agent receives only public documentation and a compiled executable binary—a black box it can run but not look inside. To succeed, the agent must independently explore the binary’s behavior, design a system architecture, write the source code, debug its own mistakes, and deliver a working build.

Refining the Teacher’s Lessons

Using a powerful teacher model (GLM-5.2), the researchers collected 1,001 end-to-end development trajectories across 562 environments spanning languages like Rust, Go, C, and TypeScript.

To ensure the training data was clean, MindForge introduced two clever refinement mechanisms:

  1. Infrastructure-Noise Recovery: If a sandbox environment crashed mid-process due to a network glitch, MindForge rewound execution to the last stable state and replayed prior steps without wasting compute.
  2. Reasoning Rewrites: If the teacher made a malformed tool call, MindForge surgically removed the mistake and rewrote the AI’s internal reasoning chain so the training trajectory remained logically coherent.

From Diagnosis to Action

The power of whole-life-cycle training becomes clear when examining how agents handle errors.

Consider a common scenario: an agent tests its code against the reference binary and notices a formatting discrepancy in a --help menu—for instance, printing command flags on one line instead of two. Standard base models frequently diagnose the issue correctly in their thought process (“I need to set the next_line_help flag”), yet immediately fall into a loop, re-running the same command dozens of times without touching the code.

MindForge dramatically changes this behavior. Fine-tuning on complete development trajectories nearly doubled the model’s discipline in converting failure diagnoses into immediate code edits—jumping from 31.8% to 48.8%.

Big-Model Performance in a 27B Package

The results were striking. When fine-tuned on MindForge trajectories, the 27-billion-parameter Qwen3.6-27B model saw its pass rate on the ProgramBench evaluation suite jump from 37.98% to 49.51%. This lightweight model matched or surpassed vastly larger frontier systems, including DeepSeek V4 Pro and Claude Opus 4.7.

Crucially, these gains generalized beyond building from scratch. MindForge-27B achieved massive improvements across seven unseen software benchmarks, including a 31-percentage-point gain on C-to-Rust code translation (RepoZero) and significant boosts on real-world issue resolution (SWE-bench).

By teaching AI agents the persistence and structure of whole-life-cycle engineering, MindForge moves open-source models one step closer to practical, autonomous software creation.