AI Papers Reader

Personalized digests of latest AI research

View on GitHub

AI “deep research” agents adopt false conclusions from a single misleading document

Deep Research agents—AI systems that plan searches, gather evidence, and write long-form reports—are increasingly used in scientific and other knowledge-intensive work. But a new study shows they are vulnerable to misinformation that looks credible. When the researchers injected one misleading document into the search results available to such systems, the average rate at which final reports endorsed a deliberately false conclusion jumped from 0% in a clean control to 54.7%.

To probe this failure mode, the team built MisKnow-Agent, a framework that generates task-specific documents supporting manually audited false conclusions. For 100 tasks from the DeepResearch Bench, it produced 5,933 such documents, varying the source’s apparent institutional authority and presentation style: academic paper, news article, technical blog, or social post. The researchers measured a false-conclusion adoption rate (FCAR)—the percentage of completed reports that endorsed the false claim as their own answer, rather than merely quoting or refuting it.

One representative document, in paper style, claimed that a 2024 release of the chemistry software Gaussian 23 introduced a keyword that automatically resolves electric-field orientation problems in single-atom catalyst simulations, improves simulation accuracy by at least 17.3%, was endorsed by a 2025 Nature Catalysis review, and had been adopted by 92% of theoretical catalysis groups. The claims were manually audited and confirmed false. The study evaluated two open-source frameworks, DeerFlow and WebThinker, each paired with three language models—DeepSeek-V4 Pro, Qwen3.5-397B, and Intern-S1-Pro—alongside the closed-source Gemini Deep Research system.

Adoption depended on when and how the misleading content appeared. Injection immediately before final synthesis produced an 85.5% mean FCAR, versus 40.5% when it entered before research began and 44.2% mid-research—increases of 45.0 and 41.3 percentage points, respectively. Source cues mattered: high-authority sources drove 61.0% adoption versus 46.2% for low-authority ones, and paper-style documents (61.0%) beat social posts (37.5%) by 23.5 percentage points. Search-result rank, by contrast, mattered little: front, spread, and back placement produced 66.2%, 65.5%, and 64.5% adoption. More documents beyond the first did not consistently add risk; FCAR stayed between 58.3% and 61.0% for two to five documents. Gemini Deep Research showed broadly similar patterns.

The authors caution that this is a controlled simulation, not a measure of how often such content appears online; the documents were inserted into isolated retrieval pools rather than published. They tested two defenses: a verification prompt issued before research, and a search-enabled refinement agent that checks the final report claim by claim. Both reduced FCAR, but neither eliminated it, and combining them did not consistently help: for Intern-S1-Pro under DeerFlow, the combined defense produced 62% adoption, higher than either defense alone (57% and 58%).

More striking, the researchers note, was that all five independent search-enabled verifier models classified every retained document as misleading when checked in isolation—yet the agents still absorbed their conclusions during long-horizon research. The authors argue that verification must be woven into evidence gathering, intermediate research states, and final synthesis—not applied only at the end. As these tools increasingly inform science and professional analysis, the study suggests a single plausible-looking false source can be enough to flip a system’s final answer.