Pretrained speech models improved for noisy police radio audio
Pretrained ASR Pseudo-labeling for Noisy Police Audio
Artificial Intelligence
Summary
Speech recognition systems struggle to understand noisy police radio recordings, making it hard to analyze police decisions. The authors studied a way to improve these systems without expensive human labels by using machine-generated transcripts (pseudo-labels). They found that usual confidence scores cannot reliably pick good transcripts from bad ones and introduced a new method using large language models to filter out unlikely sentences. They also tested using pseudo-labels from one model to train another, showing this approach could help future improvements.
What this means in practice
- •For law enforcement analysts: Improve transcription accuracy of noisy police radio recordings to support analysis of police decision-making.
- •For speech technology engineers: Develop better ASR systems by filtering pseudo-labels with language models and cross-model training to handle noisy audio.
Authors
Kaavya Chaparala, Su Huang, Stephen L. Miller, Rhiannon N. Miller, Anjalie Field
Abstract
Pretrained ASR systems perform poorly on noisy Broadcast Police Communication (BPC), hindering efforts to understand police decision-making. Pseudo-labeling offers an unsupervised path to improve ASR without expensive human labels, but the efficacy of this approach on very noisy domains is not known. In this work, we systematically assess the opportunities and limits of pseudo-labeling to adapt foundation ASR models (Whisper and Qwen3-ASR) to noisy BPC domain corpora from Baltimore and Chicago. We demonstrate that existing internal confidence metrics (log-probabilities and STAR scores) fail to distinguish between high and low quality BPC pseudo-labels, and we introduce an external LLM-as-a-judge filtering paradigm that leverages parametric knowledge to discard contextually implausible transcripts. Our LLM-judging filters more aggressively than internal metrics and significantly reduces WER of the pseudo-labeled training sets across the Baltimore and Chicago BPC corpora, though a substantial gap remains relative to an oracle filter. We also introduce a new cross-model pseudo-labeling paradigm where one model is finetuned with pseudo-labels from the other, and we identify this method as a promising direction for future pseudo-labeling work.