Synthesizer sound matching becomes faster and more accurate without rendering
Synth-JEPA: Joint Embedding Prediction for Renderer-Free Synthesizer Parameter Search
SoundMachine Learning
Summary
Matching sounds made by synthesizers usually needs testing many different settings by actually generating the sound, which takes time. The researchers created a new way called Synth-JEPA that learns a shared understanding of sound and synthesizer settings together. This allows guessing how good a setting is without making the sound first, speeding up the search for the right settings. Their method works better than older ones on known and new sounds, and people also liked the matched sounds more in listening tests.
What this means in practice
- •For audio software developers: Enable synthesizer presets to be matched quickly to desired sounds without slow audio rendering during search.
- •For sound designers: Speed up creating synthesizer patches by reducing the need for trial-and-error audio rendering.
Authors
Ben Hayes, Haokun Tian, Stefan Lattner
Abstract
Sound matching can be formulated as optimizing synthesizer parameters against an audio-domain objective. However, objectives derived from generic audio representations are often difficult to optimize, while direct search requires rendering every candidate. We introduce Synth-JEPA, which learns mutually predictive audio and parameter representations from paired synthesizer data. At inference, candidate parameters are scored directly in this learned space, yielding a renderer-free objective whose audio geometry is shaped by parameter correspondences rather than generic audio similarity. We evaluate Synth-JEPA on Surge XT using held-out synthesizer sounds and out-of-domain NSynth and FSD50K targets, against inverse models, direct search, and learned proxy objectives. Synth-JEPA outperforms all baselines in-domain and remains competitive out-of-domain. Its matching quality continues to improve with additional test-time search, allowing compute to be traded for match quality. In pairwise listening tests, listeners preferred Synth-JEPA in 85% of trials overall. Together, these results show that an audio representation with a parameter-induced geometry allows synthesizer sound matching to be approached as an effective renderer-free search problem.