AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Swarms of Latent Agents Speed Up AI-Driven Program Search

Researchers at Brown University have proposed a faster and cheaper way to use language models to search for better computer programs on mathematical and scientific problems. The approach targets a bottleneck in a growing class of tools that use large language models (LLMs) to improve code over many rounds.

These tools, known as LLM-guided evolutionary search, work somewhat like selective breeding. They generate candidate programs, score them with an automatic evaluator, and refine the strongest ones. According to the authors, the process is slow because each candidate requires the model to write a long chain of text reasoning, and that generation must be repeated many times to keep the population of ideas diverse.

The new framework, called SwarmEvolve, reduces that cost by having agents reason in “latent space,” meaning the model feeds its own internal hidden-state vectors back into itself rather than producing words. Each round launches 10 swarms of 45 agents. Each swarm has a focus, such as checking a program’s formulation or trying a textbook method, and each agent reads one parent program from an archive. Every agent then performs 40 latent reasoning steps. A novelty filter keeps the 30 agents whose reasoning trajectories are farthest apart in cosine distance, a measure of the angle between vectors, and discards 15. Finally, a summarizer agent reads the stored internal memory (key-value caches) of the kept agents and writes one new program in text. The authors describe this as a form of crossover: ideas from agents that studied different parents are combined in latent space.

The paper offers one illustration. In a circle-packing problem, the agents of one exploration swarm formed distinct clusters according to which parent program each had read, while one group read no program at all. The filter kept agents from every cluster, which the authors interpret as keeping proposals varied.

The team compared SwarmEvolve with three existing frameworks, OpenEvolve, ShinkaEvolve, and CodeEvolve, on 21 problems. Each run had 60 minutes on one GPU. Scores are relative: each problem’s objective is divided by the construction AlphaEvolve published, so a value above 1 beats AlphaEvolve. With Qwen3 8B, SwarmEvolve’s average score was 0.804, compared with 0.693 for OpenEvolve, 0.642 for ShinkaEvolve, and 0.700 for CodeEvolve. SwarmEvolve also had the highest average with Qwen3 14B and Qwen3 32B-FP8. Its mean output in 60 minutes was 734 programs, against 158, 99, and 81 for the three baselines. With Qwen3 8B, the median time to match each baseline’s 60-minute score was 4 to 8 minutes. On three problems, SwarmEvolve with the 8B model exceeded AlphaEvolve’s published constructions, including a packing of 26 circles in a square that the authors say matches the best known sum of radii.

The results have clear caveats. SwarmEvolve did not reach some baseline targets within an hour, and all agents and the summarizer must share one model, which limits mixing models. The method also requires standard attention architectures, and the memory needed for key-value caches grows with the number of agents, reasoning steps, and layers, so the largest model tested needs a large GPU. The authors also report that replacing latent reasoning with text reasoning lowered scores, but the comparison is an internal experiment rather than an independent benchmark.

The work suggests that large-scale program search does not necessarily require frontier models or very large budgets. Whether the speed and quality gains hold across other problems, models, and hardware remains to be tested.