Vision language model improves autism screening from home videos
A Deterministic Evidence Layer for Vision-Language Autism Screening from Naturalistic Home Video
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Autism diagnosis depends on experts watching children, which can be hard to access early. The authors found that existing video-based AI models give unstable decisions about autism. They built a system where the AI first lists observable events with timestamps and labels, then a separate step scores these events to give a final autism risk category. This approach is more reliable across repeated runs and works well on videos taken at home by caregivers of preschool children.
What this means in practice
- •For pediatric healthcare providers: Use natural home videos and the deterministic evidence layer to improve autism screening accuracy outside clinical settings.
- •For digital health platform developers: Integrate stable vision-language model outputs combined with evidence scoring for remote behavioral health assessment tools.
Tested on one dataset.
Authors
Wenqi Li, Mindi Ruan, Chuanbo Hu, Shuo Wang, Xin Li
Abstract
Autism spectrum disorder (ASD) is diagnosed through specialist observation of a child's social behavior, and access to that expertise is the bottleneck for early identification. Vision-language models (VLMs) describe a child's behavior from video well; the verdict drawn from the description is unstable: at temperature~0, across eight pipeline configurations on one backbone, 16--37\% of clips change their predicted label between repeated runs, and the cause lies in the serving stack. We keep the VLM frozen and move the decision out of the model. Under our grounded perception constraints the VLM writes an event table of timestamped, glossary-labeled events that names the eliciting press and logs counter-evidence; a text-only stage age-calibrates the confidence of each row; a deterministic weight-of-evidence scorer sums it into an evidence total and stratifies it into a risk category, so every decision decomposes into named per-feature contributions and can be re-scored from the saved table. On 43 caregiver-recorded, protocol-free home free-play clips of preschool children, the pipeline reaches AUC $0.851 \pm 0.012$, 86.0\% accuracy, and F$_1$ 71.8 over three runs. It labels 74.4\% of clips correctly in every run (60.5\% for the zero-shot baseline) and flags no typically developing clip in every run (9 of 31 at zero-shot). An ablation on the same backbone attributes the gain to the grounded perception constraints read through the deterministic scorer.