Pitch aligned music features enable better structure matching and transitions
Perceptually Motivated Alignment and Interpolation of Pitch-Aligned Time-Frequency Representations
Sound
Summary
Understanding how music notes relate in pitch and harmony is tricky because musical keys and scales can vary and overlap. The authors developed new methods that line up and smoothly blend musical features based on how we perceive consonance, without needing to guess the musical key first. They also designed a way to represent scales in fewer dimensions, helping to separate tone quality and variety. Their techniques work for both audio and symbolic representations, making music analysis and transformation more perceptually meaningful.
What this means in practice
- •For music software developers: Create tools that automatically align and interpolate music segments respecting harmonic structure without key detection errors.
- •For digital audio workstation makers: Integrate perceptually motivated pitch alignment into audio editing to enable musically smooth crossfades and transitions.$Commercial implications: Enhances DAWs with advanced music-aware editing features that improve user experience and output quality, providing a market differentiator.
Authors
Shahan Nercessian, Jeff Sontag, Alejandro Koretzky
Abstract
We propose a framework for the alignment and interpolation of pitch-aligned time-frequency representations. Building on the tonal interval vector, we introduce a series of extensions that reformulate it as an invertible operator, culminating in a new feature extractor that embeds perceptual consonance priors within a pitch-aligned representation. Accordingly, we develop methods for aligning and interpolating between musical structures. For alignment, we cast the problem as a permutation search under perceptually weighted distances, enabling robust matching without explicit key detection, which is inherently ambiguous in music information retrieval. For interpolation, we employ optimal transport to generate musically meaningful transitions between pitch distributions, proposing a circular formulation over the Circle of Fourths/Fifths to respect harmonic structure. Finally, we learn a low-dimensional geometric representation of scale structure aimed to factorize scale color and density. Experimental results demonstrate the effectiveness and musicality of the proposed methods. Together, our contributions provide a principled, purely signal processing approach to modeling pitch structure in audio and symbolic music signals.