Frozen model skill layer boosts audio fraud detection accuracy
FRAUDSkill: Structured Frozen-Weight Skill Optimization for Audio Anti-Fraud Detection
SoundComputation and Language
Summary
Detecting fraud from spoken audio is tricky because the system has to follow strict rules about how it decides if a call is fraudulent and what type of fraud it is. The authors found a way to improve these decisions without changing the main speech understanding model by adding an external layer that specializes in different fraud-related tasks. Their approach, called FRAUDSkill, makes fewer mistakes and works better than simply retraining the whole model. This method also adapts more easily when fraud patterns change over time.
What this means in practice
- •For call center security teams: Improve fraud calls detection accuracy without retraining speech models by adding an external skill layer that adapts to evolving fraud types.
- •For voice biometrics system designers: Integrate structured skill programs to enforce decision protocols in audio-based fraud checks, enhancing system reliability under changing conditions.
- •For financial fraud analysts: Use the skill-layer approach for adapting fraud detection models to new fraud scenarios rapidly, improving response times to attacks.$Commercial implications: Enables sale of adaptable fraud detection modules tailored to financial institutions' evolving threats.
Authors
Chengxian Hu, Zhiming Ma, Mingjun Pan, Yifan Wang, Shun Zhang, Qifan Wang, Zhilei Zhao, Yijin Zhou, Yuxi Zhao, Huiyuan Liu, Peidong Wang, Peng Chen
Abstract
Large audio-language models have shown promise for anti-fraud detection by directly processing speech and reasoning over fraud-related evidence. Their deployment, however, requires predictions to follow a predefined label space and a structured decision protocol consisting of service-scenario identification, fraud detection, and conditional fraud-type classification. Existing fine-tuning and prompt-based approaches typically encode task knowledge, constraints, and decision rules into model parameters or manually maintained prompts, making them difficult to adapt as fraud patterns and labeling policies evolve. To this end, we propose FRAUDSkill, a structured frozen-weight adaptation framework that leaves the underlying audio-language model unchanged while optimizing an external layer of skill programs, route-specific policies, and decision rules. We further combine structured output control with validation-guided multi-path inference to ensure protocol-compliant predictions. On the TeleAntiFraud benchmark, FRAUDSkill achieves 73.50% Macro-F1, outperforming the shared frozen-model baseline by 31.96% while reducing invalid outputs to 1.94%. Extensive experiments demonstrate that external skill optimization provides an effective and adaptable solution for structured audio anti-fraud detection without modifying the underlying model. The source code is available at https://anonymous.4open.science/r/FRAUDSKILL-114514.