AI Papers Reader

Personalized digests of latest AI research

View on GitHub

VISTAR: Making AI Reasoning Visible with Step-by-Step Explanations

A new approach called VISTAR (Visually Interpretable Subtask-Aware Reasoning Model) is enhancing the reasoning abilities of AI models used for Visual Question Answering (VQA) tasks. Unlike current AI models that often provide answers without clear explanations, VISTAR enables AI to break down complex questions into a series of simpler, visually-grounded steps, making its decision-making process more transparent and understandable.

Imagine being shown a picture of a beach scene with a blue tent and being asked, “Which side is the blue tent on?”. Current AI models may simply answer “Left” without revealing how they arrived at that conclusion. VISTAR, however, would deconstruct the question into subtasks such as:

  1. Identify tents: Locate all objects that could be tents.

  2. Filter by color: Select only the tents that are blue.

  3. Determine position: Compare the location of the blue tent with the center of the image to ascertain which side it is on.

  4. Answer: Output “Left”.

For each subtask, VISTAR not only provides a textual description but also highlights the relevant objects in the image using bounding boxes, visually demonstrating which areas the AI is focusing on. This is known as “object-level visual grounding.” For instance, a bounding box could be drawn tightly around the blue tent to clearly highlight the object.

One of the key advantages of VISTAR is that it integrates this step-by-step reasoning directly into the AI model through a technique called “Subtask-of-Thought (SoT) rationales.” Instead of relying on external modules, VISTAR refines existing MLLMs to produce structured SoT rationales (step-by-step reasoning sequences). This approach makes the AI more efficient and accurate compared to other interpretable methods.

The researchers tested VISTAR on two VQA benchmarks. The results demonstrated that VISTAR consistently improved reasoning accuracy while maintaining interpretability. The code and dataset are publicly available at: https://github.com/ChengJade/VISTAR.

This research offers a promising step towards creating AI systems that are not only accurate but also trustworthy and understandable, fostering greater confidence in their decisions.