Early detection of agitation in autistic youth using wearable sensors
Detecting Agitation Before Behavioral Escalation in Autistic Youth Through Multimodal Wearable Sensing
Human-Computer InteractionMachine Learning
Summary
Many autistic young people experience challenging behaviors like aggression or self-injury that start with a feeling called agitation. Agitation is hard to notice because it varies by person and includes hidden body responses. The authors collected movement, heart rate, and sounds using wearable devices during therapy sessions to spot signs of agitation. Their computer model could detect agitation fairly well just as it began and even a little before, mainly by analyzing voice sounds. This approach uses one shared model for all individuals instead of customizing one for each child.
What this means in practice
- •For clinical care teams: Provide tools to monitor autistic youth continuously and alert caregivers or clinicians to early agitation before behaviors escalate.
- •For assistive technology developers: Design wearable-based systems that use multimodal sensor fusion to detect agitation in real time to support intervention.$Commercial implications: The paper enables commercial wearable devices that alert schools or care providers about agitation onset with competitive accuracy using a single shared model.
Authors
Nibraas Khan, Abigale Plunk, John Staubitz, Ingrid Shragge, Jordan Brooks, Suzanne Wright, Alec Brewer, James Dieffenderfer, Alper Bozkurt, Amy Weitlauf, Nilanjan Sarkar
Abstract
Challenging behaviors including aggression, self-injury, and property destruction are observed in 68% of autistic youth and pose risks to youth and caregivers. These episodes are preceded by agitation, a rising state of distress expressed through movement, vocalization, and autonomic arousal. Its signs are subtle and individualized, and its autonomic components are invisible without instrumentation. We collected upper-body movement from inertial measurement units, physiology from a wrist-worn device, and vocalizations from lapel microphones across 30 clinician-led sessions with 15 autistic youth, paired with expert behavioral annotations. We adapt four pretrained foundation models, one per modality, project each to a shared 128-dimensional space, and fuse them into a single group model. The model detected agitation with an area under the ROC curve of 0.724 at the clinician-annotated onset (within-participant permutation p=0.0005), declining to 0.608 at 30,s before onset. Thirteen of fifteen participants were above chance. A from-scratch configuration reached only 0.58, while frozen and fine-tuned features performed comparably (0.71 and 0.72). Audio contributed most of the signal, and a watch-only configuration stayed near chance. Individualized agitation is therefore detectable, including in unannotated windows preceding the annotated onset, using foundation-model transfer with one shared model rather than one per child.