Mobile apps use deeplinks to speed up tasks and fallback taps
From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks
Artificial Intelligence
Summary
Smartphone apps often use taps and swipes to complete tasks, which can be slow if the app requires many steps. The authors found a way to jump directly to app pages using special links called deeplinks, reducing those steps. They combined these deeplinks with taps for actions the links can't do, creating a hybrid system. Their tool, GUI-Hopper, checks which deeplinks work on real phones and describes where they lead, making mobile automation more efficient.
What this means in practice
- •For mobile app developers: Integrate verified deeplink catalogs to speed up automated testing and task completion in their apps.
- •For automation engineers: Use hybrid interaction combining deeplinks and GUI actions to improve task success rates on real devices.
Authors
Yuchen Sun, Chenglin Cai, Gongjie Zhang, Tianyu Xia, Quyu Kong, Panrong Tong, Zhengwen Zeng, Long Chen, Steven Hoi, Chongyang Zhang, Yue Wang
Abstract
Mobile GUI agents complete tasks using GUI actions like taps and swipes. These actions are broadly applicable across applications, but reaching a navigation interface. A single deeplink call can replace a sequence of screen-by-screen GUI actions. We therefore introduce hybrid interaction, using deeplinks for direct navigation and GUI actions for other on-screen operations and fallback. To enable this, we discover candidate deeplinks through static analysis, validate them on real devices, and describe their observed landing screens. This process creates a verified and grounded deeplink catalog that pairs each working deeplink with a description of its landing screen. Using this catalog, we introduce GUI-Hopper, a improves task success in commercial applications on real devices, further demonstrating the benefits of hybrid interaction.