AI audiologist outperforms humans in bilingual simulated tests
A bilingual AI audiologist built through rubric-guided playbook induction outperforms human audiologists in a blinded evaluation of simulated cases
Artificial Intelligence
Summary
Audiology involves checking hearing health through careful questions and tests, but real patient examples are hard to get. The authors created an AI audiologist that understands both English and Chinese, follows a detailed set of rules, and analyzes hearing test images without needing extra training. They tested it using simulated patients and found the AI did better than 17 human audiologists across many different evaluation points and languages. Their method shows how creating clear step-by-step rules can help build expert systems in medical fields with limited data.
What this means in practice
- •For audiology clinics: Integrate AI consultation agents to assist or augment human audiologists during patient intake and initial evaluation in English and Chinese.$Commercial implications: Enables development of AI-driven bilingual audiology tools that can improve service efficiency and accuracy, making products viable in clinical settings.
- •For medical ai developers: Use rubric-guided playbook induction with large language models to build expert consultation agents for medical fields with scarce data.
Authors
Linkai Li, Changgeng Mo, Hanlin Yu, Congxi Lu, Shangqiguo Wang, Matthew B Fitzgerald, Shan X Wang
Abstract
Audiology consultation requires structured history-taking, audiometric interpretation and patient-centred communication, yet real-world case material is scarce. We present a bilingual AI audiologist pairing a general-purpose large language model with rubric-guided playbook induction, multimodal audiogram interpretation and retrieval-augmented grounding, without fine-tuning the language-model backbone. Using a 21-item rubric and an AI patient simulator, we induced a 19-rule consultation policy from 73 training cases (43 English, 30 Chinese) and evaluated the system on 58 independent simulated cases (30 Chinese, 28 English) in a pre-specified, source-blinded comparison with 17 practising audiologists. The AI audiologist outperformed human audiologists on every case (58/58; mean paired $Δ$ = +1.35 on a 5-point composite, Cohen's d = 1.84, $P = 4.5 \times 10^{-20}$), on 20 of 21 rubric items and in both languages. Component ablation identified the playbook as the largest contributor, offering a practical route to specialist consultation agents in low-data medical domains.