DeepOpen improves intent classification on banking and service data
DeepOpen: Decision-Oriented Intent Classification on BANKING77 and CLINC150
Artificial Intelligence
Summary
Intent classification helps computers understand what people want when they say something, like asking a bank for help or giving commands to a service. The authors present DeepOpen, a system designed to improve this understanding specifically for banking and other service domains by using special models and training methods. They tested DeepOpen on two datasets and found it performs very well, though some improvements are small and not always certain. This work highlights the strengths and challenges in building such specialized intent classifiers.
What this means in practice
- •For customer service platform teams: Use DeepOpen configurations to improve automated intent recognition in banking customer queries, boosting response accuracy.
- •For voice assistant developers: Incorporate DeepOpen’s decision-oriented models to better classify user intents within diverse service domains like banking and general customer support.
Authors
Shuang Song Zhongqiao Yuan
Abstract
We present DeepOpen, a decision-Oriented intent-classification framework with dataset- specific systems for BANKING77 and CLINC150. On BANKING77, a single English classi- fier continued with sharpness-aware minimization (SAM) achieves 94.1234% test accuracy (2,899/3,080). This is the best observed result among three fixed SAM seeds, selected for display after test evaluation; their mean is 94.0368%. On the 150 in-scope CLINC150 classes, probability fusion of one Laya specialist and three DeBERTa-v3-large classifiers achieves 98.0222% (4,411/4,500). We describe the two configurations, reproduce the reference com- parisons displayed in the accompanying evaluation explorer, and separate headline scores from evidence about improvement. Under a uniform BANKING77 evaluation path, the SAM mean exceeds the historical checkpoint by 0.0433 percentage points, with a shared-row 95% interval of [-0.2273, +0.3247]. Thus, the continuation does not establish a reliable accuracy gain. The report documents strong observed task-specific performance, selection effects, and unresolved historical execution-path differences, rather than a new backbone or a unified algorithmic advantage.