AI Papers Reader

Personalized digests of latest AI research

View on GitHub

A Misleading Sign Can Send AI Geolocation Models Thousands of Kilometers Astray

Multimodal AI systems increasingly combine visual context with text they read in images. When a street sign agrees with the scene, the extra cue can help; when it contradicts the scene, the model must decide which evidence to trust. A new benchmark suggests many leading models resolve that conflict poorly. Researchers at Peking University, 2077AI, and HKUST built SIGNPOST-Bench around visual geolocation: given a photo, the model predicts where it was taken. Because location is continuous, the team could measure not only whether errors grew but whether predictions moved toward a fake location planted in the image.

The benchmark makes five matched versions of each photo: original; text erased; text replaced with compatible wording; unrelated wording; or adversarial wording naming a real place on another continent. The synthetic edits are localized, designed to preserve surrounding non-textual content. SIGNPOST-Bench contains 5,111 such matched groups and 25,555 image variants from four datasets. The researchers evaluated 20 multimodal large language models from seven providers with the same coordinate-prediction prompt.

Adversarial text increased median localization error from 282 km to 1,347 km, a 4.8-fold rise. Native text was usually helpful: all 20 models scored higher on original images than on text-erased versions, by an average of 9.40 points. Mean weighted localization accuracy—a metric that gives exponentially more credit to closer predictions—fell from 47.11 to 29.89, a drop of 17.22 percentage points, or about 36.6% relative. Across models, 6.5–20.1% of predictions for adversarial samples with resolvable injected targets landed within 50 km of that target. Every model showed, on average, a positive paired reduction in distance to that target relative to its text-erased baseline. Text semantics mattered: compared with erased-text images, compatible replacements reduced error by 379 km on average, while unrelated and adversarial replacements increased it by 959 km and 1,577 km, respectively.

In a Google Street View image from Virginia, researchers replaced truck lettering reading “BEKINS” and “ISUZU” with “Bondi Removals” and “Tata Motors.” Gemini-2.5-Flash, asked to lay out visual and textual evidence step by step, judged the cues consistent, trusted the text, and predicted Sydney, Australia—about 15,738 km from the true location. With a defense prompt that told it to weigh visual and textual evidence explicitly, the same model flagged the conflict, trusted the scene, and predicted Virginia, about 5.5 km away. The authors caution this two-model probing experiment is preliminary and should not be generalized.

Strong clean-image performance did not predict resistance. Qwen3-VL-30B ranked fourth in conflict robustness despite modest capability, while Seed-2.0-Pro had higher capability but lower robustness. The authors propose one possible explanation: well-known place names may have strong representations from language pretraining, allowing injected names to overwhelm visual evidence. They do not claim to have proven that mechanism.

The study has limits. Its text interventions are synthetic, generated by an image-editing model, not natural examples; the authors state the adversarial texts are intended only for benchmarking, not for misleading deployed systems. Still, the benchmark provides a controlled way to see when AI trusts a sign over the scene—and a starting point for asking whether future models can learn to arbitrate better.