Small accurate pitch estimation works fast even with noise
Low-Rank Frequency Convolution and Noise-Range Augmentation for Real-Time Pitch Estimation on Edge Devices
Machine LearningSound
Summary
Pitch estimation on small devices is hard because the method must run quickly, stay accurate with noisy input, and fit in limited memory. The authors improved their previous neural network by reducing its size by over a third without losing accuracy, even on new data sets. They found that training with a wider range of noise is more important for performance under interference than changes in the model structure. They also created a small and very fast implementation that beats popular software on several CPUs.
What this means in practice
- •For mobile app developers: Provide fast and accurate pitch detection in noisy environments on smartphones with limited memory and processing power.
- •For embedded system engineers: Deploy efficient pitch estimation algorithms on edge devices where runtime speed and small footprint are critical.
Authors
Venkat Suprabath Bitra, Homayoon Beigi
Abstract
Pitch estimation on an edge device is constrained in three ways at once. The model must be small, it must stay accurate when the input is noisy, and one frame must be produced inside the frame period. In this report the Frequency Convolution Network (FrCN) of our earlier work is factored into a low-rank form. The number of parameters is reduced by 35.9%, from 17,787 to 11,397, and accuracy is not reduced, either in domain or on two corpora the model was never trained on. The range of the noise used during training is also shown to dominate the architecture in setting how the model behaves when the interference is severe. When the training noise floor is lowered from +6.02 dB to -20 dB, 0.35 points of clean RPA50 are lost and RPA50 at -20 dB is raised from 2.44 to 22.41. This effect is about two orders of magnitude larger than any architectural effect that was measured. The semi-orthogonal constraint used in TDNN-F is found to be redundant with the normalization inside the bottleneck, and accuracy is reduced when both are applied. Whether the factorization saves time depends on the runtime: in eager PyTorch the factored model is 45% slower, while in a compiled kernel it is 15% faster. For deployment, a small C kernel was written. It needs 83 kB on disk and no runtime library beyond libc and libm. It is faster than OpenBLAS on all four CPUs that were tested, faster than ONNX Runtime by 3.8 times, and faster than PyTorch by 13 times. Its output was checked against PyTorch on 271,893 held-out frames per model, and the same pitch bin was selected on every one of them.