Papers for

voice synthesis developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Generative models boost sample diversity with Jacobian based perturbations

JIVE: Jacobian-Informed Volume Expansion for Diverse Generative Sampling

Abstract: Generative models often suffer from mode collapse and limited sample diversity. While prior works attempt to mitigate this by jointly generating a batch of samples and repelling their trajectories, these heuristics do not explicitly maximize the diversity of the resulting endpoints. We introduce JIVE, a training-free framework that enhances generative diversity by injecting velocity perturbations aligned with the leading right singular subspace of the generator's endpoint Jacobian. By leveraging this local geometric structure, JIVE provably maximizes endpoint diversity while preserving sample quality. To maintain practical efficiency, we compute these perturbation directions via matrix-free iterations rooted in classical numerical linear algebra, requiring only a small computational overhead. Across different benchmarks, JIVE boosts both pixel and feature-level diversity in few-step and one-step generation.

Sun 27 SeptMachine LearningArtificial IntelligenceComputer Vision and Pattern Recognition

The gist

Generative models that create images or other data sometimes produce very similar outputs, which limits diversity. The authors introduce JIVE, a method that adds carefully chosen tweaks to the generation process based on the mathematical properties of the model. This approach helps produce a wider variety of high-quality samples without extra training. It works efficiently by using classical numerical techniques, improving diversity in various generation tasks.

Open → 2609.33906v1

Acoustic models influence text-to-speech deepfake detection effectiveness

Tracing and Relearning Detection Evidence in Text-to-Speech Systems

Abstract: Recent audio deepfake detectors separate bona fide speech from synthetic speech, yet it remains unclear which stage of a text-to-speech system supplies the detection evidence. We address this with controlled resynthesis and detector adaptation in an F5-TTS-BigVGAN pipeline. Since vocoder reconstruction of a real mel can itself be separable from the source utterance, we fix the vocoder and trace the larger change in detector separation to acoustic generation. Adversarially fine-tuning the acoustic model, with no detector in its objective, raises EER against fixed detectors at comparable quality. However, adapting a detector only on the tuned model's VCTK outputs lowers its LibriSpeech EER from 19.42% to 7.46% and improves detection of unseen base F5-TTS outputs. These results suggest that acoustic-model updates can reduce the detection evidence available to fixed detectors, while detector adaptation keeps the updated outputs detectable in this pipeline.

Fri 25 SeptSound

The gist

Detecting fake speech generated by text-to-speech systems is important for security and trust. The authors studied which part of the speech generation process gives away clues that help detectors spot fakes. They found that changes in the acoustic model affect how easily a detector can tell real from synthetic speech. Even when the acoustic model was adjusted without trying to fool detectors, it became harder for fixed detectors to identify fakes. However, retraining detectors on the updated speech kept detection effective, showing that both acoustic model updates and detector adaptation are crucial for fake speech detection.

Open → 2609.30983v1