Multimodal AI detects mismatched ECG records with text consistency checks
SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity
Computation and Language
Summary
Medical records often combine different types of information like waveforms and text reports, assuming they all belong to the same patient. The authors identify a hidden problem when parts get mixed from different patients, which can cause safety risks. They created a lightweight method called SHIFT-M3 that checks if two text summaries about an ECG match, indicating if the record is consistent. This method works very well at spotting full and partial mismatches while being computationally efficient.
What this means in practice
- •For hospital data teams: Detect mismatched ECG records assembled from different patients to improve clinical data safety.
- •For medical ai developers: Incorporate text-based consistency checks into multimodal clinical AI systems to identify record mislinkages.
Authors
Md Ashik Khan, Md Nahid Siddique
Abstract
Multimodal clinical AI typically assumes that the waveform, report, metadata, and downstream predictions attached to a record belong to the same patient. In practice, linkage failures can silently assemble individually plausible but cross-patient components, creating a safety problem that standard predictive models are not designed to detect. We study this problem as multimodal record integrity triage: given an assembled record, should its modalities be trusted to belong together? We introduce SHIFT-M3, a lightweight text-based pre-fusion screen that measures alignment-based consistency between two separately produced ECG text views: an LLM-generated interpretation and a clinical report summary. On 784,680 MEETI ECG records, SHIFT-M3 achieves 97.6% TPR@5% FPR for full text-view swaps (AUROC 0.996), 90.3% for partial swaps (AUROC 0.974), and 97.7% for label-matched hard negatives (AUROC 0.996) with only 573,569 parameters. Compared with same-dataset lexical baselines, the gains are largest on partial swaps and hard negatives, suggesting that the model is learning more than surface overlap. We also introduce the CMST (Conflict-type Multimodal Stress Test) evaluation taxonomy, a three-seed stability study, a loss ablation, a temporal-tolerance sweep, and a shared-token masking control. The main remaining failure mode is longitudinal ambiguity: at the default operating point, same-patient cross-visit pairs still produce 87.0% Type-II false positives.