AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Intervention-Based Prompt Optimizer Targets the Right Agent in Multi-Agent AI Systems

Multi-agent artificial intelligence systems built from large language models (LLMs) can divide complex tasks among specialized agents, but their performance depends heavily on each agent’s prompt. Existing methods for automatically optimizing those prompts often use “textual gradients”—natural-language critiques that suggest how to revise a prompt—yet they can target the wrong agent or combine unrelated failure signals. AgentGrad, a new framework from researchers at Korea University, KAIST, Meta AI, and UNIST, tries to make that process more precise by identifying which agent’s correction actually fixes a failure.

The framework has two main parts. First, sequential intervention tests one agent at a time. For each failed example, AgentGrad injects a training-time hint into an agent’s prompt, in reverse execution order, to see whether that single intervention makes the multi-agent system produce the correct answer. The agent whose correction resolves the failure becomes the target. Because the intervention produces a corrected intermediate output under the same input, AgentGrad can compare it with the agent’s original output and use the difference as agent-level supervision, yielding a fine-grained textual gradient for that agent’s prompt. Second, semantic textual gradient abstraction clusters sample-level gradients that share a corrective pattern into semantic minibatches. An aggregator LLM then abstracts each cluster into one generalized gradient, aiming to avoid mixtures of unrelated failure modes. Candidate prompt updates must improve performance on their minibatch and then on a held-out validation set before acceptance.

A concrete example comes from PUPA, a privacy-conscious delegation benchmark. There, one agent must rewrite a private user query into a sanitized request for an external LLM. For a query asking for a business plan for a street food truck in Warsaw, Poland, the failed output still exposed “Warsaw, Poland.” After intervention, the adjusted output replaced the location with a placeholder and instructed the external model not to infer the real place. AgentGrad extracted a gradient to treat city and country names as sensitive, part of a broader redaction policy.

In experiments across five multi-agent benchmarks—HotpotQA, HoVer, PUPA, IFBench, and MATH—the authors report that AgentGrad achieved the largest average improvement over the no-optimization baseline, surpassing MIPROv2, TextGrad, and GEPA. With GPT-5-mini, it averaged an 11.76-point improvement over the no-optimization baseline, scoring 73.89 on HotpotQA and 95.17 on PUPA, compared with GEPA’s 68.33 and 91.87. With Qwen3-8B, it averaged a 9.67-point improvement over the no-optimization baseline. On GPT-5-mini, AgentGrad also completed optimization in 136 minutes on average, which the authors describe as 2.5 times faster than GEPA, the next-fastest baseline, and 4.7 times faster than TextGrad.

The approach still has constraints. Hints are constructed from ground truth or output constraints and are used only during training, so optimization may require labeled failures or task-specific guidance. Failures that no single-agent intervention resolves are excluded from the current training round, though revisited later. And the evidence covers five benchmarks and two backbones, leaving broader unlabeled or open-ended deployments untested.

The broader significance is that better credit assignment—knowing which agent to fix—can improve both prompt quality and optimization efficiency in multi-agent LLM systems. But the results establish benefits within the tested benchmarks and configurations, not a general solution for all multi-agent settings.