Papers for

security teams for ai systems

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language action models vulnerable to black box adversarial patches

Black-Box Adversarial Patch Attacks on VLAs via Ancestor VLM Exploitation

Abstract: Vision-Language-Action models (VLAs) are increasingly deployed in safety-critical physical environments, yet their adversarial robustness remains poorly understood. Existing attacks typically assume white-box access or rely on surrogate VLAs, which rarely holds in real-world deployments. Our key insight is that most VLAs are adapted from a publicly released pretrained vision-language model (VLM), inheriting two capabilities essential for action generation: visual perception and instruction-conditioned grounding. Therefore, this paper explores a previously unaddressed question: can an adversary attack deployed VLAs using only their ancestor VLMs? To this end, we propose three adversarial patch attacks that disrupt the inherited capabilities: a vision disruption attack that corrupts the projected visual tokens through relative and absolute terms, an instruction-grounded semantic evidence suppression attack that removes the visual evidence required for instruction-grounded concepts, and a joint attack that unifies both objectives under a two-phase curriculum. Experiments across different VLA families on both simulation and static real-world images show that patches optimized on the ancestor VLM cause substantial degradations in VLA task success rates, demonstrating that VLAs inherit adversarial vulnerabilities alongside their foundational capabilities. This effect is not uniform: it is strongest on tasks that require precise instruction-grounded localization, and nearly vanishes on policies whose adaptation rewrites the shared visual-semantic representation or whose action head iteratively smooths perturbations away. By characterizing the boundary conditions of vulnerability inheritance and providing analysis of why the inheritance effect holds or fails, we advance the understanding of safety for VLA-involved systems.

Wed 7 OctCryptography and Security
The gist
Vision-language-action models help computers see and act based on instructions, often in important environments like robots. This paper shows that attackers can fool these models using small visual patches created by attacking their shared base model, without needing full access to the deployed system. The attacks mainly work by confusing how the model sees images or matches them to instructions, especially in tasks needing precise location. This means vulnerabilities can carry over from the base model to the final action system, raising safety concerns for real-world deployments.
Open → 2610.09708v1

Adversarial attacks threaten vision language models in self driving cars

Transferable Spatial Temporal Coherence Adversarial Attack on Black-Box Vision Language Models for Autonomous Driving

Abstract: The rapid integration of Vision Language Models (VLMs) into sensitive systems introduces critical safety vulnerabilities that remain unexplored in exist studies. While adversarial attack robustness has been extensively studied for image-based models, the susceptibility of VLMs to temporally-aware adversarial attacks against video in driving context poses a distinct and under examined threat. In this paper, we introduce novel adversarial attack against video targeting VLM models used for autonomous driving scenes named Spatial Temporal Coherence Adversarial Attack (STCA). Our attack comprise from three stages: modalities expansion, Spatial attack, and STCA attack. In modalities expansion, we propose caption-guided frame selection method in order to ensure that adversarial perturbation target the most semantically significant frames. Secondly.In spatial attack, we craft effective perturbation and preserve high similarity. Then the perturbed video generated fed into STCA stage that disrupt cross-frame temporal coherence using motion guided mask. Our method operate under black box threat model against victim target VLMs, relying solely on transferability from white-box surrogate model.We conduct our experiments on the BDD100K and nuScenes autonomous driving datasets across three VLM models: Video LLaVA-7B, Qwen2.5-VL-7B, and Dolphin. Experimental results demonstrate spatial attack achieves an ASR with high SSIM. Our finding reveal that existing video language model, remain highly susceptible to adversarial attack in autonomous driving scenarios, underscoring the urgent need for robust defense for VLM models.

Tue 6 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Autonomous cars use video-based vision language models (VLMs) to understand their surroundings. This paper shows that these models can be fooled by subtle changes in video frames that confuse the system over time. The authors created a new attack that picks key video frames and changes them in ways that disrupt how the model reads the video sequence, all without knowing the model's inner workings. Their tests show current models remain vulnerable, highlighting the need for better defenses in self-driving technology.
Open → 2610.08331v1

Adversarial attack reduces concept segmentation accuracy across prompts

Universal Cross-Prompt Adversarial Attacks on Promptable Concept Segmentation

Abstract: The Segment Anything Model (SAM) achieves remarkable performance in visual segmentation. The latest SAM3 extends promptable segmentation to concept-level prediction, broadening the scope of segmentation foundation models. While recent works reveal that SAM and SAM2 are vulnerable to adversarial examples, the robustness of SAM3 under the concept segmentation paradigm remains unexplored. In addition, existing adversarial attacks on SAM-series models exhibit limited cross-prompt transferability. To this end, we propose AdvPCS, a universal cross-prompt adversarial attack for Promptable Concept Segmentation (PCS), including a min-max prompt optimization strategy, a global-local perception deception attack, and a temporal transition deviation attack. Specifically, we first identify the hardest-to-attack prompts via min-max bilevel optimization. In the inner maximization, we enhance diversity over candidate point, box, and text prompts. In the outer minimization, we select prompts with the highest responses based on the confidence scores output by the detector. Given the selected prompts, we apply the perception deception attack to minimize both global and local existence probabilities under joint prompting and employ the temporal memory misalignment attack to maximize inter-frame semantic inconsistency and corrupt memory pointers. Extensive experiments on four benchmark datasets show that a single universal adversarial perturbation (UAP) generated by our method generalizes across frames from different videos and achieves strong attack performance under point, box, and text prompts. In particular, it reduces the average mIoU of various PCS models on the SA-CO dataset to below 5% under text prompts, demonstrating strong attack ability.

Wed 30 SeptComputer Vision and Pattern Recognition
The gist
The paper looks at how well a new type of image segmentation model, SAM3, which identifies concepts in images based on prompts like points, boxes, or text, stands up against tricky changes designed to fool it. The authors propose a method called AdvPCS that creates a single pattern of small image changes, which can confuse the model no matter what prompt is used. They test this on various datasets and show it can drastically lower the accuracy of the model’s segmentations, especially when using text prompts. This shows that even advanced segmentation models are vulnerable to universal tricks that work across different ways of asking the model to segment.
Open → 2609.39265v1

Kolmogorov Arnold stability holds for discontinuous deep learning functions

Kolmogorov--Arnold stability for discontinuous functions

Abstract: Here we investigate the stability of the Kolmogorov--Arnold representation theorem (KART) under adversarial reparameterisations of the hidden layer for multivariate discontinuous and unbounded functions. Our results provide a rigorous mathematical foundation for the structural robustness of modern deep learning architectures, such as Kolmogorov--Arnold Networks (KANs), under adversarial configurations.

Mon 7 SeptMachine Learning
The gist
The paper studies how a mathematical way to represent complex functions, called the Kolmogorov–Arnold theorem, stays reliable even when the function is jumpy or unsteady. The authors explore how this stability works when deep learning networks face tricky changes that try to confuse or break them. Their findings support why certain deep neural networks remain robust when their inner layers are reconfigured in adversarial ways. This helps explain why these networks can perform well even under challenging conditions.
Open → 2609.07240v1