Papers for

user interface automation teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

BlueLM-GUI improves mobile app control using real devices and smart training

BlueLM-GUI Technical Report: A Real-Device-Centric Flywheel for Self-Improving Mobile GUI Agents

Abstract: Mobile GUI agents are shifting from multi-module frameworks to native models trained end-to-end, yet industrial deployment faces three persistent gaps. Sandbox training produces a distribution mismatch with production environments; expensive real-device failures remain underutilized; and fixed benchmarks saturate, losing the power to guide iteration. We present BlueLM-GUI, a 35B-A3B mobile GUI agent built as a real-device-centric flywheel that closes these gaps through three principles. Every Sample Matters: a dual-track pipeline with Heterogeneous Triple-System Consensus evaluation and an Error Correction \& Derivation Module salvages every trajectory into usable supervision. Every Rollout Is Real: a three-stage recipe---continual pre-training, supervised fine-tuning, and agentic reinforcement learning on hundreds of real phones---grounds every rollout in real production environments, so the capability the model learns transfers directly to deployment. Every Query Evolves: a quota-driven benchmark methodology with three orthogonal axes enables precise attribution and allows the benchmark to be systematically upgraded as the model improves. BlueLM-GUI achieves 87.4 on MobileGUI-VBench, surpassing the best closed-source model by 5.1 points, and 84.9 on AndroidWorld, the best result among open-source models and competitive with closed-source models. These results demonstrate that grounding model training and iterative improvement in both real devices and the three Every principles yields strong, robust, and transferable mobile GUI capability.

Fri 11 SeptArtificial Intelligence
The gist
Controlling mobile apps automatically with AI can be tricky because models often learn in fake environments that differ from real phones, and errors during testing are rarely used to improve them. The authors created BlueLM-GUI, a 35-billion-parameter AI model trained directly on many real phones using a special method that recycles every sample and error to keep learning and improving. Their approach includes continuous training, careful evaluation, and updating benchmarks, leading to better performance than earlier systems. This means BlueLM-GUI can better understand and interact with mobile app interfaces as they really appear.
Open 2609.12394v1