Robust Summarization of Doctor-Patient Conversations: TalTech Systems for the Beyond Transcription Challenge
Computation and Language
Summary
The authors explain how they built systems to create medical notes directly from long doctor-patient audio recordings, skipping the usual step of turning speech into text first. They improved two versions of a speech language model using special training techniques that focus on matching medical concepts correctly. Their methods won first place in both categories of the challenge and produced fewer mistakes or 'hallucinations' than others. They also found that training on text transcripts helped their models work better with real-world speech recordings.
Authors
Aivo Olev, Tanel Alumäe
Abstract
This paper describes TalTech's submissions to the Beyond Transcription Challenge (BeTraC), which requires generating SOAP notes directly from long doctor-patient conversation recordings, without intermediate transcription. After screening open-weight speech LLMs for long-audio robustness, we adapted Voxtral Mini (lightweight track) and Voxtral Small (heavyweight track) with LoRA supervised fine-tuning followed by DAPO reinforcement learning that uses the challenge metric, Open Medical Concept F1, as its reward. Our systems ranked first in both tracks, and an independent LLM-as-a-judge evaluation showed the lowest hallucination rate among all submissions, indicating that reinforcement learning against a concept-matching metric need not compromise factual reliability. We also find that fine-tuning on text transcripts transfers well to speech input and appears to improve robustness on out-of-domain real recordings.