AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Why AI Agents Fail: Bringing a Social Science Method to the Logs

As large language models take on longer tasks—fixing code, navigating websites, answering customers—they fail in ways a pass/fail flag cannot explain. A model may act confidently through many steps, then miss the one diagnostic check that mattered. Reading every step by hand is slow; classifiers built from known failure modes may miss novel breakdowns. Researchers have now automated grounded theory, a six-decade-old qualitative method in which categories emerge from data rather than prior hypotheses, to map what agents actually do.

The system, called AutoTraceGT, works as a team of LLM-powered analysts. One agent “open codes” a single trajectory, labeling each behavioral episode with a short conceptual phrase and a verbatim quote. A second agent groups those codes across batches of trajectories into behavioral categories. A manager merges or splits categories against the running codebook and keeps an append-only revision log. The loop stops at “saturation”—when new batches add almost no new categories. A final agent writes a narrative theory connecting those categories.

Using GPT-5-mini to build the codebooks, the researchers compared them with human-expert taxonomies for ALFWorld, GAIA, and WebShop. The machine-generated codebooks recovered 75.0%, 73.7%, and 90.9% of the human failure-mode categories, and they covered the human-written reasoning for individual trajectories in 58–88% of cases. The codebooks also surfaced patterns the human taxonomies missed. In ALFWorld, for instance, AutoTraceGT recorded “noncompliant inaction”: the agent produces no admissible action when one is required. The human taxonomy’s invalid_action and impossible_action labels both presuppose that an attempted action occurred, so the distinction—total absence versus a bad attempt—had been collapsed.

Downstream, the codebook showed practical use. On three outcome-labeled benchmarks—Tau-Bench, Go-Browse, and SWE-Agent—the authors used codebook features to predict failure from the first half of a trajectory. The features sometimes beat an LLM prompted directly to predict failure, but they were not always the strongest option; combining AutoTraceGT’s codebook with a few-shot codebook produced the largest gains in many settings. With GPT-5-mini on Go-Browse, for example, AutoTraceGT features achieved a Matthews correlation coefficient of 0.374 compared with 0.370 for the few-shot-codebook baseline, while the combined feature set reached 0.425. The theoretical narratives also converged with earlier expert descriptions of failures as cascading errors, though AutoTraceGT framed them as observable behavior rather than internal cognitive modules.

The authors stress limits. Every coding stage is performed by an LLM, so the categories may inherit model biases shared across frontier models; the tests rule out dominance by one model, not all models. The coverage evaluation also relied on an LLM judge, so reported percentages depend on that judge. And because the pipeline makes many LLM calls per trajectory, it is designed for offline inspection of large corpora, not real-time monitoring.

More broadly, AutoTraceGT offers an auditable route from raw logs to behavioral theory. That could help developers see repeated failure patterns in new tasks before choosing fixes, and it positions agent logs as data for qualitative inquiry rather than just benchmarks to score. The authors present the work as an analytic tool, not a claim that agent behavior has been fully decoded; the categories they generate, they caution, are hypotheses that still deserve human review.