Papers for

music production engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Joint embeddings link audio effect code with its sound output

PEACE: Joint Embeddings of DSP Effects Code and Audio

Abstract: This paper introduces PEACE, the first joint embedding of audio effect code and output audio. Building on SLAP's multimodal objective, we pair an AFx-Rep audio encoder with two code encoders for Faust, a functional language for audio signal processing. First, we evaluate a fine-tuned T5 transformer over Faust source code. Second, we evaluate a message-passing graph neural network over an intermediate representation of the Faust compiler, capturing both topology and UI parameters. We evaluate on audio-to-code retrieval, where masking UI parameters at inference yields embeddings that encode effect chain topology alone. When parameters are visible, the two code encoders tie on retrieval of mixed-length chains but have tradeoffs on single-effect galleries. With parameters fully masked, PEACE recovers ordered chain topology far above chance without the limitations of supervised methods. PEACE outperforms pretrained models on an out-of-distribution reverb retrieval benchmark and can improve frozen audio-only representations. Its dual understanding of topology and parameters lays the groundwork for music information retrieval systems that search, generate, and condition on DSP code.

Fri 2 OctSound
The gist
Matching computer code for audio effects with the sounds they produce is hard. The authors created PEACE, a technique that learns to connect audio effect code written in a special language with the sound it creates. PEACE uses two different ways to understand the code and compares them with the sound, helping it recognize how the code changes the audio. This makes it easier to search for audio effects or understand how a particular sound was made by looking at the code behind it.
Open → 2610.03405v1