AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Neurosymbolic Agent Excels at Explaining Flowchart Reasoning, Outperforming LLMs

College Park, MD – A new study from researchers at the University of Maryland, Adobe Research, and Arizona State University introduces a novel approach to interpreting flowcharts using a neurosymbolic agent that significantly outperforms existing vision-language models (VLMs). Flowcharts, critical for visualizing decision-making processes across various domains, pose a challenge for standard VLMs due to their non-linear structure and complex visual-textual relationships, leading to hallucinations and unreliable interpretations.

The researchers address this issue with FlowPathAgent, a system designed for fine-grained flowchart attribution. This means it can trace the specific components of a flowchart that justify a given response from a language model, ensuring verifiability and improving explainability.

“Imagine you’re using a flowchart to troubleshoot a technical issue,” explains Manan Suri, a researcher at the University of Maryland and lead author of the study. “If the system suggests a particular solution, FlowPathAgent can show you exactly which path through the flowchart led to that recommendation.”

To achieve this, FlowPathAgent employs a three-step process. First, it segments the flowchart into distinct components and converts it into a structured symbolic graph. Then, an “agent” dynamically interacts with the graph, leveraging graph-based reasoning to generate “attribution paths.”

For example, consider a flowchart outlining steps for preparing for a potential vehicle breakdown. A question might be: “What is the immediate next step after utilizing prepared items for seeking help, and what decision led to this step?” Instead of relying on potentially hallucinated information, FlowPathAgent traces the path:

  • Starting at “Utilize Prepared Items for Seeking Help”.
  • It identifies the immediately preceding decision: “Need to Leave Vehicle?”.
  • Then it identifies the correct next step based on a “Yes” decision.

The researchers also introduced FlowExplainBench, a new benchmark dataset for evaluating flowchart attribution across diverse styles, domains, and question types. Experimental results show that FlowPathAgent significantly mitigates visual hallucinations in LLM answers over flowchart question-answering (QA), outperforming strong baselines by 10-14% on the FlowExplainBench dataset. This highlights FlowPathAgent’s ability to understand conditional pathways that define flowchart semantics.

Suri emphasized the importance of this research, stating, “In critical applications like healthcare or software verification, it’s crucial to know why a system made a certain decision. FlowPathAgent provides the necessary transparency and reliability for deploying flowchart-based systems in these high-stakes domains.”

The researchers plan to release the code and data associated with FlowPathAgent and FlowExplainBench to facilitate further research in this area.