Mobile agents face growing security risks from evolving jailbreak attacks
MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile Agents
Cryptography and Security
Summary
Mobile agents that can move and act on your phone are becoming smarter, but this makes them easier to trick or misuse. The authors found that current tests for security problems don’t fully check how these agents behave across real phone tasks. They created MobileJailBench, a tool that constantly updates its tests to find new ways attackers could trick these agents, showing that current protections are often weak. This helps people better understand and protect mobile agents from harmful misuse.
What this means in practice
- •For mobile app developers: Evaluate their AI-driven mobile apps against evolving jailbreak methods to improve app security and user safety.
- •For mobile security teams: Use the dynamic benchmark to identify and patch vulnerabilities in mobile agents before attackers can exploit them.
Authors
Songping Wang, Borui Li, Qinglong Liu, Yueming Lyu, Yi Huang, Fengting Li, Chao Feng, Caifeng Shan
Abstract
MLLM-based mobile agents are rapidly gaining autonomy in perception, planning, and closed-loop execution in dynamic mobile environments. However, this autonomy also broadens the attack surface and amplifies the risks of malicious misuse, including privacy leakage, device compromise, and large-scale phishing. Existing jailbreak evaluations leave three critical gaps for mobile agents: they focus on conversational attacks instead of mobile workflows, use static task sets with fixed difficulty, and rely narrowly on attack success rate rather than comprehensive security assessment. To close these gaps, we present MobileJailBench, the first evolving jailbreak benchmark for mobile agents, comprising three key components: (1) a benchmark comprising 520 tasks spanning three jailbreak paradigms and 14 risk categories, built on a high-fidelity mobile sandbox that we construct, enabling realistic workflows across over 20 widely used apps; (2) Dual-Dynamic Evolution, which combines Failure-Distilled Jailbreak Evolution with Weakness-Driven Benchmark Expansion; and (3) Stratified Adjudication Framework for assessing security awareness, execution behavior, and risk consequences. Extensive experiments show that current mobile agents are particularly vulnerable to implicit jailbreaks, as their safety alignment remains largely superficial and fails to generalize from conversational refusal to dynamic mobile interactions. Moreover, our dynamic evolution mechanism sharpens practical threat boundaries by continually uncovering new vulnerabilities, reaching a 100% ASR on Qwen3.5-122B. MobileJailBench provides a reproducible, continuously evolving framework for tracking the true security boundaries of mobile agents.