AI Papers Reader

Personalized digests of latest AI research

View on GitHub

Fake or Real? New AI Model Spots Synthetic Images with Detailed Explanations

The rise of AI-generated content (AIGC) has blurred the lines between real and fake, making it increasingly difficult to discern authentic images from synthetic ones. To combat this growing problem, researchers at the Shanghai Artificial Intelligence Laboratory and other institutions have developed a new large multimodal model called FakeVLM. Unlike existing methods that primarily focus on binary “real” or “fake” judgments, FakeVLM goes a step further by providing clear, natural language explanations for the artifacts it identifies in synthetic images.

“The goal is to improve the transparency and trustworthiness of synthetic image detection,” says Weijia Li, a researcher at Sun Yat-Sen University and a corresponding author of the paper. “Instead of just saying an image is fake, we want to explain why it’s fake.”

To train and evaluate FakeVLM, the team created a comprehensive dataset called FakeClue, containing over 100,000 images across seven categories, including animals, humans, objects, scenery, satellite images, documents, and deepfakes. Each synthetic image in FakeClue is annotated with fine-grained artifact clues in natural language. For example, if FakeVLM identifies a fake image of Venice, it might explain: “This image is a fake because the buildings have a strange connection structure with the water surface, the bridge has a distorted structure, and the chimney in the distance has blurred edges.”

The researchers highlight that FakeVLM focuses on detecting artifacts generated by synthetic image models, as opposed to forgery artifacts that arise from manipulating existing images. These synthetically-generated artifacts often include structural inconsistencies, distortions, and implausible physical characteristics.

FakeVLM’s architecture builds upon the LLaVA model, integrating a vision encoder, multimodal projector, and a large language model. This allows the model to analyze images and generate coherent textual explanations.

Extensive evaluations across multiple datasets demonstrate FakeVLM’s superior performance in both authenticity classification and artifact explanation. In some cases, FakeVLM even surpasses human-level accuracy in identifying fake images.

Concrete Examples:

  • Animal Images: In a fake image of a zebra, FakeVLM might point out “The zebra’s stripes are inconsistent… some areas appearing overly smooth and others too rough; the texture of the zebra’s skin is inconsistent.”
  • Satellite Images: For a synthetic satellite image, FakeVLM could identify that “The outlines of buildings and vehicles are blurry and distorted. The boundaries between vegetation, land, and buildings are blurred, with texture mixing and unnatural transitions…”
  • Document Images: FakeVLM could reveal that “The handwritten content suddenly appears amidst document. The text contains nonsensical phrases that fail to meet readability standards; the layout is misaligned…”

By providing human-readable explanations, FakeVLM enhances trust in AI-based authenticity assessments, making it a valuable tool for combating misinformation and fraud. The researchers plan to release the dataset and code publicly, hoping to foster further research and development in this critical area. You can find the code and dataset here: https://github.com/opendatalab/FakeVLM.