Latent audio diffusion model adapted for endless guqin music playback
Adapting a Latent Audio Diffusion Model to Historical Guqin Recordings: A Listening-Driven Case Study
Sound
Summary
Creating AI that can play guqin music continuously is challenging because of limited historical recordings and the instrument's unique sounds. The authors took a pre-trained AI model and fine-tuned it on a curated set of old guqin solos to generate new music in the same style. They tested different approaches, found what worked best for producing convincing continuations of the music, and identified issues that needed fixes. Although the results are not statistically conclusive, the study provides useful insights into adapting AI music models to rare instruments.
What this means in practice
- •For audio software developers: Create AI-driven music generators that continue guqin pieces convincingly using small curated datasets of historical recordings.
- •For cultural preservation teams: Develop tools to generate new musical content reflecting traditional guqin styles for archival or educational use.
Tested on one dataset.
Authors
Huanchen Cai
Abstract
We report a small-data case study in adapting a pretrained latent audio diffusion model to the guqin, the seven-string Chinese zither, aiming at an "AI radio" that plays guqin-style music without end. From a library of historical recordings we curate 412 solo performances (42.5 h, 61 performers) and split them by composition. We fine-tune a rank-16 DoRA adapter on Stable Audio 3 Medium using its continuous latents, masking weighted towards continuation, captions that combine researched notes on each piece with mood tags and an automatically estimated pentatonic mode, and random-length crops, stopping when held-out loss stops improving. Seven blind listening studies by one expert listener guided every decision. The final adapter was rated highest for continuing unseen pieces (3.9/5, against 3.4 for the best earlier adapter) and 4.6/5 for generating from free-written scene descriptions. Negative results are equally informative: a from-scratch autoregressive model over the same latents produced no recognisable timbre, a signal-level friction-noise measure correlated with the listener's complaints in the wrong direction, and a pentatonic-fit measure tracked ratings overall but barely within a group of candidates. Chaining continuations for long playback exposed a silent tail on every generated clip and a loudness feedback loop, both with simple fixes. With one listener and at most ten clips per condition, no paired difference is statistically significant; we present an exploratory record of what helped, what did not, and why.