Tandem reduces latency and energy for AI GUI agents on mobile NPUs
Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUs
Artificial Intelligence
Summary
Smartphone AI agents help users by reading app screens and suggesting actions, but running these AI models on phones is slow and uses a lot of battery. The authors present Tandem, a system that makes these models run faster and use less power by smartly reusing previous computations and managing the phone's special AI hardware more efficiently. Tandem works by saving parts of the AI processing so it doesn't have to start over each time, which helps it respond quicker and keep the phone's energy use down. The authors tested Tandem on recent Snapdragon chips and saw noticeable speed and battery improvements.
What this means in practice
- •For mobile app developers: Run AI-based smartphone agents locally with lower delay and battery use, improving user experience in apps.
- •For mobile chipset engineers: Optimize mobile NPUs to handle interactive AI tasks efficiently by adopting Tandem’s memory and power management techniques.
Authors
Zhuomin Chen, Jingchao Ni, Xu Zheng, Yong Chen, Mo Sha, Wei Cheng, Dongsheng Luo
Abstract
Smartphone GUI agents complete tasks in mobile apps by repeatedly reading the screen and asking a model for the next action. Mobile neural processing units (NPUs) enable local execution, keeping screen contents on the phone, but model latency accumulates with every action. Hybrid linear attention models reduce memory requirements, yet reusing their recurrent state requires checkpoints at the appropriate input positions. Agent requests retain instructions but rewrite recent history, so checkpoint policies designed for chat continuation can miss reusable prefixes. Retaining rollback states for speculative decoding can also move prompt processing off the NPU's matrix unit, adding overhead that outweighs decoding savings on short action outputs. A loaded model can further hold the NPU awake between tasks. We present Tandem, a runtime for hybrid GUI agents on mobile NPUs. Tandem saves recurrent state where consecutive prompts diverge, allowing later requests to reuse previously processed context. It keeps most prefill on the matrix unit and produces rollback states through a short per-token tail. Its verification kernel reads each weight block once for all proposed tokens. Between request bursts, Tandem releases NPU power votes without unloading the model. We implement Tandem in llama.cpp and evaluate it on recorded agent interactions across two Snapdragon generations. On a Snapdragon 8 Elite, Tandem reduces per-step latency by 30% for text-based screen descriptions and 16% for screenshots relative to llama.cpp's fastest NPU configuration, reduces energy per text step by 33%, and lowers standby power with the model loaded from 0.76 to 0.22 W.