Cloud-Boosted Low-Compute Multi-Channel Speech Enhancement
2026-08-07 • Sound
SoundMachine Learning
AI summaryⓘ
The authors address the problem of improving speech quality on devices like earbuds that have very limited computing power and need to work in real time. They build a system where a more powerful server helps the small device by sharing extra information in three ways: sending delayed outputs, sharing intermediate data layers, and combining statistical information to improve sound filtering. Their experiments show this teamwork between server and device leads to much better audio enhancement without making the device work much harder. This method is smarter than just relying on the device alone.
Speech enhancementLow-latency processingEdge computingKnowledge BoostingMultichannel Wiener filteringBeamformingFeature transferReal-time communicationCollaborative inferenceCovariance matrices
Authors
Xulin Fan, Juan Azcarreta, Ashutosh Pandey, Jesus Alvarez, Ke Tan, Jacob Donley, Ritwik Giri, Buye Xu
Abstract
Low-latency, low-compute speech enhancement is essential for wearable devices with real-time communication requirements, but strict computational constraints significantly limit on-device performance. Knowledge Boosting has been proposed as an effective approach to improve edge model performance by leveraging a more capable server-side model, but performance gains for speech enhancement have been limited. We propose a collaborative framework incorporating three techniques: (1) delayed server output as additional input, (2) layerwise feature boosting that transfers intermediate server representations to guide edge inference, and (3) collaborative multichannel Wiener filtering, which fuses weighted covariance matrices estimated from both server and edge models for improved beamforming. Experimental results demonstrate that the proposed collaborative framework significantly outperforms the edge-only baseline with minimal additional computational overhead.