Voice phishing detection compares feature types for better results
Comparing Self-Supervised and Domain-Invariant Features for Cross-Domain Voice Phishing Detection
SoundComputation and Language
Summary
Detecting voice phishing calls is tricky because real recordings are hard to get and there aren’t many examples to learn from. The authors tested two ways to spot these calls: one using simple speech patterns that don’t change across different recordings, and another using advanced self-learning models that understand voice features deeply. They found that simple speech patterns work okay without any real examples, but the advanced models do much better when they have some real data to learn from. This shows a balance between needing less data and getting higher accuracy in spotting phishing calls.
voice phishingself-supervised learningdomain-invariant featuresHuBERTwav2vec2.0cross-domain evaluationprosodic featureszero-shot learningfew-shot learningF1 score
Authors
Jeongmin Lee, Seung Yun, Minkyu Lee, Ran Han, Yoonkyu Woo, Jinxia Huang
Abstract
Voice phishing detection faces three critical challenges: real criminal recordings are unavailable due to privacy constraints; when available, only a handful of samples exist, insufficient for fine-tuning; and lightweight acoustic-only detection is needed as an alternative to large self-supervised models. We compare domain-invariant prosodic features and self-supervised representations (HuBERT, wav2vec2.0) through cross-domain evaluation-training on scenario-based actor recordings and testing on authentic criminal calls. Domain-invariant prosodic features achieve 69.5% F1 zero-shot and 71.0% with 5-shot learning. HuBERT achieves highest performance (94.2% F1, 5-shot), while wav2vec2.0 exhibits a precision-oriented detection profile (90.2% F1 with 99.4% precision, 5-shot). These findings reveal fundamental trade-offs: domain-invariant features enable zero-shot deployment when no real data exists, while SSL methods achieve higher performance but require real samples and compute.