Small language models improve calls screening and summarizing on device
Quality-Controlled Reply and Summary Data for Post-Training Small Language Models in AI Call-Screening Agents
Computation and Language
Summary
Phones can now answer calls for you without sending your private information to the cloud, but small language models struggled to reply properly and summarize calls accurately. The authors created a way to train these models with carefully controlled synthetic conversations and quality checks. After training, models got much better at replying appropriately during calls and giving useful summaries afterward. This progress reduces mistakes like false alarms and incorrect statements in summaries, though there is still work needed to ensure accuracy about who said what.
What this means in practice
- •For mobile phone developers: Build on-device call screening features that reply appropriately and summarize calls without sending user data off the device.$Commercial implications: Enables phone makers to offer privacy-respecting AI call assistants that generate reliable replies and summaries, a commercially attractive feature for consumers.
- •For customer support platform teams: Improve AI agents that handle customer calls by training small models to provide consistent replies and accurate call summaries for review.
Authors
Jiaqi Gan, Tsang Ng, Ankit Raj, Tommy Duong, Xingyu Shen, Kidus Zewde, Huaiyu Guo, Simiao Ren
Abstract
Small language models have improved to the point where a phone can plausibly screen its own calls, answering on an absent owner's behalf without sending call content off the device, a privacy-respecting alternative to cloud call handling. Realizing that promise takes a model that can do two things well: reply within the owner's authority during the call, and afterward tell the owner what happened and what needs attention. Released small models do neither reliably - they raise unwarranted alarms about ordinary callers and write summaries that drop material conditions or report a caller's claims as fact - and real calls supply no reference replies or summaries to learn from. We construct supervision for both tasks from synthetic calls under an information-boundary constraint: a caller writer works from a private scenario brief, a reply writer sees only the conversation up to the current turn, and a summary writer sees the completed transcript. Separate model-based reviews of the full dialogue and the summary decide which calls enter training, yielding a corpus of 3,395 calls with 15,102 targets (11,707 reply, 3,395 summary). Training a single shared LoRA adapter on both tasks, we evaluate six models (0.6-4B parameters) before and after adaptation on 192 held-out calls. Reply qualification rises from 21-54% to 94-97% and summary qualification from 0-6% to 43-72%; all twelve paired gains (six models x two tasks) have bootstrap intervals above zero, and the summary gains hold in every source family. Because one model both screened the training data and scored the outputs, these rates measure agreement with that evaluator's standard rather than human acceptance, which a planned human study will test. After adaptation, coherence and naturalness failures nearly vanish, leaving grounding and usefulness - faithfully tracking who said what, under which conditions - as the remaining challenge.