Papers for

voice security teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Voice recognition models evaluated against human perception of cloned voices

VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning

Abstract: Speaker-recognition benchmarks test how accurately a model recovers the speaker label of a recording, treating voice identity as a property of the person who produced it. However, these benchmarks do not evaluate how humans perceive voice identity. Whether a listener hears two voices as the same person has consequences for intellectual property rights, cybersecurity, and everyday life. We introduce the Voice Identity Perception benchmark (VIPBench), which contains 124,876 same-or-different judgments from 1,290 listeners on 9,800 recording pairs from 100 public figures. The pairs include real recordings, voice clones generated by a text-to-speech system, and morphs between two speakers' voices. We also define four tasks that evaluate the alignment between speaker-representation models and human identity perception. The first two tasks measure how well a model's similarity score predicts the proportion of listeners who respond "same" and the majority response. The other two measure whether a model and listeners find the same speaker pairs similar, and whether a model's relationship to listener judgments holds equally for real and synthetic recordings. We evaluate eleven publicly available speech models. Listener judgments diverge from speaker labels most often for voice clones: in 41% of responses, listeners judged a clone and a real recording of the same speaker to be different people. No model reaches the noise ceiling; the best model explains 61% of the explainable variance. Models trained with speaker labels align with listeners better than self-supervised models do, yet a model with a 12% error rate on speaker labels aligns with listeners as well as models with error rates below 2%. VIPBench enables the evaluation of speaker models against human perception on real, cloned, and morphed speech, and our results motivate training speaker models directly on listener judgments.

Tue 3 NovSoundArtificial IntelligenceComputers and Society
The gist
Voice recognition tests usually check if a computer can identify who is speaking in a recording, but they don’t see if this matches what humans hear. The authors created a large test, VIPBench, where over a thousand people judged if pairs of real or computer-generated voices sounded like the same person. They found that humans often disagree with voice labels, especially for voice clones created by machines. The best computer models still don’t perfectly match human judgments, showing room for improvement by training models using human listening data.
Open → 2611.05638v1

Replayfactor improves detection of replayed deepfake speech attacks

ReplayFactor: Replay-Aware Dual-Attribute Learning for Practical Speech Deepfake Detection

Abstract: Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.

Mon 2 NovSound
The gist
Deepfake speech that is played back with physical devices can fool many security systems. The authors designed a method called ReplayFactor that uses multiple learning techniques to better recognize these replayed fake voices. They also created a new large dataset of replayed fake Chinese speech to train and test detectors. Their approach significantly improves accuracy and reduces errors compared to older methods, helping systems detect these more realistic voice fakes.
Open → 2611.02628v1

Speech tokens enable realistic voice imitation with chosen text

From Speech Tokens to Stolen Voices: Attacker-Chosen-Text Speaker Impersonation via Representation Inversion

Abstract: Speech interaction systems have gradually evolved from traditional cascaded systems that use text as intermediate information to end-to-end speech language models that directly process speech representations. In some applications, the speech frontend runs on the user's device and sends speech representations to a remote server for inference, while the original audio remains local. However, these representations may still contain speaker identity information. If an attacker obtains them, it may use this information to imitate the user's voice, threatening their voice privacy. To investigate this risk, we propose \stia, a Speech Token Inversion Attack that generates speech with attacker-specified text in a victim's voice using a single short leaked representation, without access to the original audio or its transcript. Specifically, \stia converts discrete, continuous, or mixed speech representations into conditioning sequences and trains a conditional flow-matching speech generation model using pairs of different utterances from the same speaker in public speech datasets. The model combines speaker characteristics contained in the leaked representation with the specified text to generate speech in the victim's voice. Experiments show that, with a Moshi representation leak corresponding to 3 seconds of speech, \stia achieves an average CAM++ speaker cosine similarity of 0.78 and a target-text word error rate of 3.80\%. These results show that speech frontend representations reveal reusable speaker identity information and highlight the need to consider privacy risks when designing speech frontend representation methods.

Mon 26 OctSoundArtificial IntelligenceCryptography and Security
The gist
Speech systems often send hidden speech information instead of raw audio to protect privacy. The authors show that these hidden speech codes still carry enough voice details for attackers to mimic someone's voice speaking any chosen sentence. They built a tool that, given just a short leaked speech code, can generate new speech in the victim’s voice without needing the original recording or transcript. This reveals potential privacy risks in how speech data is shared and stored.
Open → 2610.34688v1

WavLM layers reveal how speech models detect fake audio attacks

Interpreting SSL Representations for Spoof Detection: a WavLM Study

Abstract: Self-supervised speech models such as WavLM achieve strong spoofing detection performance, yet which layers encode the most spoof-relevant information and what acoustic properties they capture remain opaque. We present a layer-wise interpretability study of WavLM using the ASVspoof 5 database, combining three complementary analyses: (i) per-layer linear probing with attack-level evaluation across speech synthesis, voice conversion, and adversarial attacks; (ii) a canonical correlation analysis (CCA) linking layer representations to spoofing-relevant acoustic features; (iii) an exhaustive grid search over layer subsets showing that use of five layers outperforms full thirteen-layer pooling. CCA analysis reveals that, while most acoustic features peak in alignment with mid-layers and then decay, voice quality measures jitter, shimmer, and the harmonics-to-noise ratio show the opposite trend, with correlation increasing toward later layers. Embedding visualisations reveal that deeper layers capture a continuous acoustic similarity among attacks rather than discrete categories, suggesting these layers encode fundamental synthesis artifacts rather than attack-specific signatures.

Fri 16 OctSoundMachine Learning
The gist
Detecting fake or spoofed speech is important for security, but it's unclear how self-supervised speech models like WavLM do it. The authors studied which layers of WavLM best detect various fake speech attacks and what sound features they focus on. They found mid layers capture general audio traits while later layers detect signs related to voice quality. These deeper layers don’t just spot specific attacks but recognize subtle artifacts common across different types of fakes.
Open → 2610.23526v1

Quantum circuits improve fairness in deepfake speech detection confidence

QBFF: Deep Quantum Bottleneck for Automatic Speaker Verification Anti-Spoofing

Abstract: A central goal of quantum machine learning is to identify applications in which small quantum circuits can improve task performance while remaining executable on real hardware. We investigate deepfake (synthetic) speech detection, a largely unexplored application in this area, where reliable confidence scores are as important as discrimination performance because deployment decisions are made at fixed thresholds and overconfident errors can be costly. We introduce QBFF, which routes a compressed speech representation through a small 8-qubit variational circuit (VQC) used as a drop-in bottleneck, with a matched classical bottleneck as the control. The quantum circuit yields better-calibrated confidence in most matched comparisons, including out-of-distribution evaluation, though the downstream deployment-cost benefit is model-dependent. We then run the same circuit on a real IBM quantum processor. To our knowledge, this is the first anti-spoofing pipeline to run on a QPU; hardware execution demonstrates feasibility, with lower subset EER and $C_{llr}$ but higher mean actDCF than the matched classical reference. Together, these results show that hardware-executable quantum bottlenecks can improve score reliability, with gains that vary by evaluation setting.

Mon 12 OctMachine Learning
The gist
Detecting fake or synthetic speech is important to stop misuse, but it needs not just accuracy but trustworthy confidence scores. The authors introduced a small quantum circuit as part of a deep learning model to better judge how sure the system is about its decisions. This quantum part improved the reliability of confidence scores, including when tested on unusual or new types of fake speech. They even tested the quantum part on a real quantum processor, showing it can work outside simulations, with some mixed results on different measures.
Open → 2610.16390v1

TTS speech watermarks improve traceability under signal changes

GAMW: Generation-Aware Mel-Domain Watermarking for TTS Traceability

Abstract: As text-to-speech (TTS) systems continue to advance, provenance tracking and security verification of synthesized speech have become increasingly important. Audio watermarking provides active provenance by embedding imperceptible, machine-recoverable information into speech. However, existing post-hoc methods typically modify the waveform after synthesis, leaving the watermark vulnerable to compound perturbations. To address this limitation, we propose GAMW, a watermark injection framework integrated into the TTS generation process. Before waveform synthesis, GAMW embeds a content-adaptive multi-bit watermark residual into the generated representation, and its decoding objective combines focal modulation with dynamic per-bit weighting. We train and evaluate GAMW separately on two TTS backbones. With a 16-bit payload, GAMW improves decoding accuracy over the strongest baseline by 9.2 and 23.5 percentage points under individual and compound perturbations, respectively, while maintaining competitive speech quality. Reliable clean-speech decoding is also retained at higher payloads. Code is available at https://github.com/ZConJugaTe/GAMW.

Sat 10 OctSoundArtificial IntelligenceCryptography and Security
The gist
Text-to-speech systems create spoken audio from text, but it can be hard to track who made the speech and verify its origin. The authors propose a method called GAMW that hides a secret digital code inside the speech as it is made, rather than adding it afterward. This makes the hidden code more reliable even if the audio is altered or affected by noise. Their approach improves the accuracy of detecting this code while keeping the speech quality natural.
Open → 2610.15082v1

Speaker verification systems tested across diverse real-world scenarios

OpenSVBench: Diagnosing Speaker Verification Generalization Across Deployment Scenarios

Abstract: Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization. VoxCeleb-style evaluations remain useful anchors for English in-the-wild media speech. Scenario-specific evaluations and challenges are also valuable. However, neither setting directly shows whether the same system remains reliable across diverse deployment scenarios. We introduce OpenSVBench, a public benchmark for cross-scenario SV evaluation. Its fixed public core contains 26 trial sets organized into 11 scenario groups, with 9.15M verification trials, frozen target/non-target keys, and metadata for scenario-wise reporting. Around this core, the online leaderboard acts as a diagnostic evaluation interface rather than only a ranked list. It accepts score packages from new systems, evaluates them with frozen keys and common scoring, and reports trial-set breakdowns, scenario-group metrics, macro EER/minDCF, model-wise profiles, and scenario-wise comparisons. We evaluate 18 public checkpoints spanning x-vector, ECAPA-TDNN, CAMPPlus, ResNet and ERes variants, ReDimNet2, SAM-ResNet, and pretrained Transformer speaker recipes. The results show clear cross-scenario gaps: systems with similar macro EER can have different scenario profiles, strong performance in one scenario does not guarantee strong performance in another, and same-scenario trial sets can still expose source and protocol gaps.

Fri 9 OctSound
The gist
Speaker verification systems can identify who is speaking, but they often work well only in certain situations. This paper introduces OpenSVBench, a public benchmark designed to test how well these systems perform across many different real-world scenarios. The authors tested 18 different systems and found that good performance in one situation does not guarantee good performance in others. This helps show where current technology may still struggle in real use cases.
Open → 2610.13760v1

Adaptive purification strengthens audio deepfake detection against attacks

Detection-Guided Adaptive Purification with Diffusion Models for Robust Audio Deepfake Detection

Abstract: Audio deepfake detectors remain vulnerable to adversarial perturbations that suppress the acoustic cues used for detection, allowing manipulated utterances to evade the detector. Although existing defenses can improve robustness, they require retraining the detector or introduce additional distortion. Diffusion-based purification instead leaves the pretrained detector unchanged, but existing methods use the same purification strength for all inputs, creating a trade-off between removing adversarial perturbations and preserving the subtle spoofing cues needed for detection. In this paper, we propose Detection-Guided Adaptive Purification (DGAP), a diffusion-based defense that adjusts purification strength per input. Building on the observation that a light purification perturbs the detector score of an adversarial input far more than that of a benign one, the framework uses the resulting score shift as a reference-free indicator of adversarial manipulation. Inputs with small shifts are passed unchanged, whereas flagged inputs undergo stronger purification before final detection. We evaluate the framework against three adversarial attack settings across three deepfake detectors, and compare it with nine existing defenses. Our results show that DGAP achieves the best defense performance across all detectors while leaving benign inputs nearly unaffected, and remains effective under the defense-aware adaptive attack.

Wed 7 OctCryptography and SecuritySound
The gist
Audio deepfake detectors can be tricked by subtle changes that hide fake voices. The authors designed a smart cleanup step that adjusts how much it 'cleans' each recording based on how suspicious it looks. If a recording is likely fake, it gets a stronger cleanup to remove tricks; if not, it stays mostly unchanged. This approach helps find fakes more reliably without messing up real voices. They tested it on several detectors and attacks and found it works better than other ways to defend against trickery.
Open → 2610.10752v1

Acoustic and content attacks impact multilingual voice anonymization effectiveness

Exploiting Acoustic and Content-Oriented Speaker Verification Attacks Against Multilingual Voice Anonymization

Abstract: Attacker ASV systems for voice anonymization have been studied primarily in English, leaving their behavior in multilingual settings largely unexplored. Conventional ASV has shown that both acoustic and contextual information are important for multilingual speaker verification. Inspired by this, we investigate whether the same holds for attacker ASV on anonymized speech. We evaluate both acoustic- and content-oriented attackers on multilingual anonymized speech and construct a multilingual voice-converted dataset to improve cross-lingual generalization. Our results show that attacker effectiveness depends on the linguistic utility of the anonymized speech. Overall, acoustic-oriented attackers achieve better performance. However, when linguistic information is well preserved, the performance gap between content- and acoustic-oriented attackers narrows compared with conditions involving stronger speech distortion. The multilingual voice-converted dataset further improves performance and partially reduces the cross-lingual gap. These findings highlight the need for more comprehensive attacker modeling and evaluation protocols that consider both privacy and utility, rather than relying on a attacker strategy\footnote{Full code and pretrained models and MultiVC Dataset link are available at: https://github.com/monkeyDarefeen/DAST

Tue 6 OctSoundArtificial Intelligence
The gist
Voice anonymization tries to hide who is speaking while keeping the message understandable. The authors studied how attacks on voice anonymization work across different languages. They found that attacks focusing on sound features generally work better, but when the voice still sounds clear and content is preserved, attacks focusing on speech content get closer in success. They also created a new multilingual voice dataset that helps attackers across languages do better. This shows that protecting voice privacy requires understanding both the sound and the speech meaning.
Open → 2610.08107v1

Partial speech spoofing detection improves false alarm control but challenge remains

Temporal Anchors and Editing Sensitivity in Partial Speech Spoofing: A Controlled Study

Abstract: Partial-spoof detectors must reject synthetic content while accepting benign edits. We diagnose temporal anchors and boundary-consistency training using frozen WavLM features, nonlinguistic unit controls, and source-locked evaluation. Genuine-genuine and genuine-fake splices contrast editing false alarms with synthetic-content misses; they do not isolate a unique causal artifact. With layer-6 features, phone units have higher PartialSpoof evaluation frame EER than soft frames (11.18% versus 9.59%). In 500 retrospective PS-eval cases, augmentation reduces genuine-splice false alarms; synthetic-core misses increase at 5% PS-development FPR but not demonstrably at 1%. A 200-case Llama A follow-up retains the false-alarm reduction but does not establish a miss-rate increase. Constructed-case ranking can improve while fixed-threshold misses rise. Consistency gives no uniform gain, and encoder-layer rankings change across corpora. We report frame and event localization with three seeds, including word units, and treat PS evaluation as retrospective. Audit code, selected training scripts, and result summaries are available at https://github.com/mysxs/partial-spoof-diagnostics.

Sat 3 OctSound
The gist
Detecting when parts of a speech recording have been faked is tricky because systems must spot fake parts while ignoring harmless edits. The authors studied how timing clues and training strategies affect this detection using special voice features. They found that some ways to train reduce false alarms on real edits but may slightly increase missed fake detections. The system’s performance also varies depending on the data and settings used. Their work includes tools and data to better understand and improve these partial spoofing detectors.
Open → 2610.04479v1

Speech deepfake detector improves spotting fake voices on new sources

GLAD: Global-Local Adaptive Detector for Robust Speech Deepfake Detection

Abstract: Recent advances in AI-based speech synthesis have enabled highly realistic speech, increasing the importance of speech deepfake detection (SDD) in preventing misuse. While mainstream Self-Supervised Learning (SSL)-based detectors achieve strong performance, they suffer from poor generalization to unseen domains and often overlook fine-grained signal artifacts due to a bias towards global semantic consistency. In this paper, we conduct the first detailed empirical and visual analysis to validate these limitations explicitly. Our investigation reveals two critical architectural vulnerabilities: (1) a systemic failure to capture localized spoofing traces, and (2) a severe lack of adaptability to domain-driven shifts in SSL layer importance, rendering static aggregation strategies prone to overfitting. To address these vulnerabilities, we propose the Global-Local Adaptive Detector (GLAD). Specifically, to capture localized forgeries, GLAD employs a Hierarchical Global-Local (HGL) backbone that explicitly bridges the granularity gap by fusing global linguistic and acoustic features with fine-grained local signal details. To counter layer importance shifts in out-of-distribution (OOD) scenarios, we introduce a Hierarchical Adaptive Gating (HAG) mechanism that dynamically recalibrates layer-wise focus in a sample-specific manner. Finally, to address shortcut learning induced by environmental biases, we introduce SaniBoost, a composite data augmentation strategy for robust signal standardization and noise sanitization. Extensive experiments demonstrate that GLAD significantly outperforms state-of-the-art methods, particularly on unseen domain cases.The code will be released upon publication.

Mon 28 SeptSoundArtificial Intelligence
The gist
Realistic fake voices made by AI can trick people, so detecting them is important. The authors found that existing detectors struggle with voices from new sources and miss subtle fake signals. They designed a new system called GLAD that looks closely at both overall speech patterns and tiny details to better spot fakes. GLAD also adapts to different kinds of voice recordings and uses data tricks to avoid being fooled by background noise. Their tests show GLAD works better than current methods, especially with unseen types of fake audio.
Open → 2609.35411v1

Speech deepfake detection improves by fixing training conflicts

DGS-MLDG: Domain Gradient Surgery Guided Meta-Learning for Domain Generalization in Speech Deepfake Detection

Abstract: Speech deepfake detection faces significant challenges due to domain shifts. Domain generalization (DG), particularly meta-learning for domain generalization (MLDG), offers a promising solution by simulating and mitigating domain shifts. However, MLDG is often hindered by conflicting gradients between its meta-train and meta-test objectives, leading to suboptimal performance. To address this problem, we propose domain gradient surgery (DGS), a meta-learning method that resolves conflicts through an asymmetric projection strategy. DGS removes the destructive component from the meta-test gradient, ensuring a conflict-free optimization trajectory versus the meta-train gradient. Furthermore, we introduce layer-wise DGS (LW-DGS), an efficient variant of DGS that dynamically identifies and intervenes only conflict-prone layers. Extensive experiments on challenging benchmarks demonstrate that DGS-MLDG and LW-DGS-MLDG achieve an average relative EER reduction of 5.29% and 4.04%, respectively.

Sun 27 SeptSound
The gist
Detecting fake speech can be tricky because the way people speak changes in different situations. The authors found that one popular way to train detectors runs into problems because two parts of the training disagree and slow progress. They created a new method that stops these disagreements from messing up training. This helps the system get better at spotting fake speech even when the speaking style changes.
Open → 2609.33706v1

Speech deepfake detection improves using combined acoustic features

What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection

Abstract: The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly those relying on speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes. We therefore compare 12 acoustic representations using a shared low-capacity linear classifier to identify effective evidence under codec shift. The analysis shows that hierarchical XLS-R leads on the pooled test set, while pooled no-vocals residual statistics perform best on the unseen-codec condition, revealing complementary behavior across generation conditions. Building on this finding, we propose MN-P, a dual-view detector that integrates an utterance-level pooled no-vocals representation with token-level XLS-R features through adaptive gating. The proposed MN-P reduces EER by 54.2% overall and by 60.9% on the codec-unseen condition relative to the best-performing retrained state-of-the-art system, with consistent gains across different detector backends. These results indicate that pooled no-vocals residual statistics provide effective complementary evidence for cross-generation speech deepfake detection.

Sun 27 SeptSoundArtificial IntelligenceMachine Learning
The gist
Detecting fake speech is getting harder because new ways to create it sound more realistic. The authors studied many sound features to find which ones best spot fakes across different speech generation methods. They found that a mix of two types of features, including one called no-vocals residuals, works especially well when the fake speech method is new or unseen. Combining these features into a single detector greatly reduced errors in spotting fake speech. This approach could help build stronger protections against audio deepfakes.
Open → 2609.33375v1

Graph-based wavelet tool cuts parameters for speech deepfake detection

WST-Graph: Topology-Preserving Wavelet Scattering Front-End for Speech Deepfake Detection

Abstract: The acoustic front-end determines which forensic cues a speech deepfake detector can exploit. The wavelet scattering transform (WST) provides stable multiscale coefficients with explicit coordinates, yet direct flattening obscures the parent relation between paths. We introduce WST-Graph, reconstructing these paths as a sparse modulation-carrier grid for an AASIST graph backend. Modulation-level normalization and length-aware adaptive local attention pooling produce fixed relative-time representations while retaining the acoustic axes before learned adaptation. This yields a waveform-to-graph interface with a fixed, parameter-free WST. Our configurations remain competitive with AASIST while using approximately 60% fewer trainable parameters and show clear gains on selected out-of-domain benchmarks. These results underscore the value of preserving parent-child relations within the carrier-modulation topology when constructing a compact, physically grounded interface for graph-based speech deepfake detection. Code will be released at https://github.com/saki-ciallo/wst-graph.

Thu 24 SeptArtificial IntelligenceSound
The gist
Detecting fake speech can be tricky because the part of the system analyzing sounds needs to focus on useful details. The authors created a new way to organize wavelet scattering data, keeping the connections between sound components in a graph structure. This lets their method capture important speech clues using fewer adjustable parts while still working well, especially when tested on unfamiliar examples. Their approach helps computers better spot faked voices by keeping the sound data’s natural organization.
Open → 2609.29372v1

Audio deepfake detection improves across languages using language orthogonalization

Language Orthogonalization for Zero-Shot Cross-Lingual Audio Deepfake Detection

Abstract: Audio deepfake detectors need to transfer to languages absent from training, as multilingual speech synthesis outpaces labeled anti-spoofing resources. While detectors increasingly rely on self-supervised speech models (S3Ms), these backbones encode language-dependent structure that confounds spoof cues. We address this confound through language orthogonalization, a target-free ridge map that removes S3M variation projected onto continuous language-identification (LID) embeddings. Across six languages, six S3M backbones, and all Leave-N-Out settings, it consistently reduces EER across unseen languages. Cross-lingual EER correlates with LID-space distance, where orthogonalization yields larger gains for more distant transfers.

Tue 15 SeptComputation and Language
The gist
Detecting fake audio is hard, especially when the audio is in languages that the system wasn’t trained on. The authors found that current speech models mix language-specific patterns with signs of fakeness, confusing the system. They developed a way to separate the language part from the fake detection part, improving detection for languages not seen during training. This method works better when the new language is very different from those in the training set.
Open → 2609.16458v1