Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation
2026-07-27 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionCryptography and Security
AI summaryⓘ
The authors created a new method to detect fake GPS signals that could trick self-driving cars by combining camera views and car sensor data like speed and direction. They used a special model that links images and words to spot inconsistencies between what the GPS says and what the car's other sensors show. They tested this on real driving data with different types of GPS lies and found their method was much more accurate than previous ones. They also made the system faster by only checking some of the time. This work adds a practical extra layer of safety for autonomous cars beyond just checking the GPS signals alone.
Autonomous VehiclesGNSS SpoofingVision-Language ModelsSensor FusionLocalizationIMUTrajectory MirroringZero-shot LearningF1-scoreReal-world Dataset
Authors
Mohammed Aldeen, Muhammad Sami Irfan, Sagar Dasgupta, Long Cheng, Mizanur Rahman, Mashrur Chowdhury
Abstract
Autonomous vehicles (AVs) depend on Global Navigation Satellite Systems (GNSS) for localization and navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first Vision-Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.