Procedural audio pre training improves with source scaling and mask tuning
Rethinking Procedural Audio Pre-training: Source Scaling and Objective Adaptation
SoundArtificial Intelligence
Summary
Procedural audio, which is artificially generated sound, can help teach machines to understand audio better. The authors studied how to best create and use these sounds to teach machines by looking at two things: how varied the sound formulas are and how much variety there is in the sounds they produce. They found that different ways of making these sounds help in different learning tasks, and that settings used for regular audio don't always work well with procedural audio. Their work suggests adjusting training methods depending on the kind of audio source being used.
What this means in practice
- •For audio machine learning engineers: Improve audio pre-training models by tailoring procedural audio source diversity and mask ratio settings for better downstream task performance.
- •For game audio developers: Generate diverse procedural audio sets optimized for machine learning to create richer sound effects adaptable to different game scenarios.
Authors
Jiajun Peng, Fengrui Liu, Xinyu Liu, Feng Liu
Abstract
Procedural audio has emerged as a viable source for transferable audio representation learning, but its design principles remain unclear.We revisit two questions: how a procedural source should be scaled, and whether training choices developed on natural audio should transfer unchanged to procedural data.Using a controlled source, we separate scale into formula-class coverage C and within-class rendering diversity I.Experiments with FDSL and AudioMAE show that these two forms of scale provide different benefits and depend on the learning formulation and downstream task. A matched AudioMAE study further shows that procedural audio favors low mask ratios (10%--25%), whereas AudioSet-28K favors 50%--75%. Shared-codebook analysis reveals lower patch diversity and stronger temporal predictability in procedural audio. These results motivate source-aware procedural pre-training, where source scaling and learning configuration are considered jointly.Code is available at https://github.com/Cross-Innovation-Lab/Formula-Bank.