Transcript compression reduces accuracy of detecting medical fake videos
Effects of Transcript Compression on LLM-based Medical Misinformation Detection in Japanese YouTube Videos
Computation and Language
Summary
Detecting misinformation in long medical videos is important but challenging. This study found that using full transcripts of Japanese medical YouTube videos helps language models better identify fake content. When transcripts are shortened by summarizing or selecting parts, models make more mistakes, especially missing fake videos. The shortened versions also change the language signals, making fake videos seem more believable and harder to catch.
What this means in practice
- •For video content moderators: Use full transcripts instead of summaries to improve detection of medical misinformation in long videos on platforms like YouTube.
- •For health communication platforms: Design tools that avoid transcript compression so that automated checks better catch misleading medical videos.
Tested on one dataset.
Authors
Yuya Wake, Sho Tsugawa, Toshiyuki Amagasa
Abstract
Large language models (LLMs) are increasingly used to assess long-form medical videos, but their effectiveness may depend on whether transcripts are provided in full or compressed through summarization, retrieval, or claim screening. This study examines how such transcript compression affects LLM-based veracity classification of Japanese medical YouTube videos. We compare four transcript input designs: full transcripts, LLM-generated summaries, RAPTOR-based retrievalaugmented generation (RAG), and Screening, which extracts candidate medical and health-related sentences. Using 74 long-form videos labeled as Real or Fake, we evaluate classification performance and analyze linguistic changes using J-LIWC, hedge expressions, and institutional or technical terms. The full-transcript Baseline achieved the best performance, whereas all compressed inputs increased false negatives, meaning that Fake videos were more likely to be misclassified as Real. Summary caused the largest performance drop, while Screening performed best among the compressed inputs but still omitted many medically relevant sentences. Linguistic analyses showed that these errors were not explained by a simple increase in certainty. Instead, Summary reduced affective, social, temporal, cognitive, and conversational cues, while Summary and RAG made institutional and technical terms more salient. These findings suggest that transcript compression can represent Fake videos as more coherent and authoritative inputs, thereby weakening cues needed for misinformation detection