Pitch correction improves by using accompaniment notes as context
MIDIBack: Harmony-Aware Singing Pitch Correction via Joint Vocal-Accompaniment Symbolic Modeling
Sound
Summary
Correcting singing pitch automatically is tricky because it's hard to tell if a singer is off-key or just purposely changing notes for expression. The authors created a method called MIDIBack that looks at both the sung notes and the accompanying music notes together to fix pitch more accurately. Their method works better than ones that only focus on the singer's voice because it understands the harmony context. This makes the corrected singing sound more natural and in tune with the music.
What this means in practice
- •For music software developers: Integrate joint vocal and accompaniment note modeling to improve automatic pitch correction in singing software.
- •For karaoke machine manufacturers: Enhance pitch correction features by using accompaniment information to produce more natural-sounding singing corrections.$Commercial implications: This paper enables accompaniment-aware pitch correction products that improve karaoke user experience by producing harmonically accurate vocal tuning.
Authors
Joaquim Cavalcante, Yicheng Gu, Adriel Trajano, Yuri de Malheiros, Thais Gaudencio
Abstract
Automatic pitch correction (APC) requires distinguishing the unintended intonation errors from expressive pitch variation. Existing systems either lack explicit harmonic modeling, as vocal-only methods do, or do not directly use the note-level polyphonic context. Therefore, we propose MIDIBack, a note-level APC framework that jointly models the vocal and accompaniment events in a shared OctupleMIDI sequence. We evaluate MIDIBack under 6 note corruption regimes, including global outshift, learned note-dependent detuning, uniform perturbations, and their combinations. The resulting model achieves 78.6% overall raw pitch accuracy (RPA), and 81.5% under combined global outshift and learned detuning. Removing the accompaniment conditioning reduces RPA from 81.5% to 35.8% in outshift, showing the effectiveness of accompaniment context. Case studies on accompaniment modulation further illustrate that vocal note predictions