AI summaryⓘ
The authors present BareWave, a new text-to-speech system that directly generates sound waves from text without relying on intermediate acoustic steps. They identify three main challenges in this direct approach: lacking a pretrained framework, needing different training noise settings at different stages, and difficulty aligning perceptual features with model training. To address these, they introduce training techniques that align representations, adjust noise schedules, and incorporate velocity-aware perceptual alignment, all while keeping inference simple and native to waveforms. Their experiments demonstrate that this method can produce clear and natural speech, even for new voices, without extra pretrained parts.
Text-to-Speech (TTS)Waveform-nativeFlow-matchingNoise schedulingPerceptual alignmentZero-shot voice cloningVelocity-space flowRepresentation alignmentDirect waveform modeling
Authors
Wei Fan, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Kejiang Chen, Weiming Zhang, Nenghai Yu
Abstract
Removing intermediate representations and separately trained decoding stages has become an important direction in generative modeling. In text-to-speech, however, high-quality systems are still commonly built through an intermediate acoustic representation before waveform synthesis. In this work, we present BareWave, a fully waveform-native framework for direct text-to-wave generation in flow-matching TTS. We consider this setting to raise three training challenges: raw-waveform modeling lacks a strong pretrained representational scaffold, different stages of training benefit from different noise schedules, and data-space perceptual objectives do not automatically share the temporal structure of the velocity-space flow objective. As a result, direct waveform training is hard to optimize efficiently, hard to push toward a strong final operating point with a fixed recipe, and hard to integrate effective perceptual refinement. Guided by this view, we develop a direct text-to-wave training framework that combines training-time representation alignment, staged noise scheduling, and velocity-aware perceptual alignment (VAPA), while preserving a single waveform-native inference path without pretrained components at test time. Experiments on zero-shot voice cloning show that strong intelligibility, speaker similarity, and naturalness can be achieved under a fully waveform-native inference path, supporting waveform-native flow-matching TTS as a practical direction. Project page with audio demos is available at https://barewave.github.io/.