Confidence filtering affects accuracy of video-based gait assessments
Confidence Filtering in Video-Based Gait Assessment: Measurement Error versus Selection Shift
Computer Vision and Pattern Recognition
Summary
Video systems that analyze walking patterns often ignore low-confidence data points to reduce measurement mistakes. The authors compared this approach to random data selection in people with Parkinson’s and another condition. They found that filtering by confidence can slightly change the movement results, mixing true errors with selection bias. Filtering helps only if confidence relates mostly to error, but if confidence is linked to the movement itself, it can shift the results and introduce bias. In their Parkinson’s data, confidence had little relation to movement, so filtering was mostly harmless if unbiased windows were kept.
What this means in practice
- •For clinical motion analysis teams: Adjust gait data selection methods to minimize bias when using confidence scores for pose filtering in neurological conditions.
- •For medical device developers: Incorporate findings about confidence filtering impacts into video-based gait assessment tools to improve measurement reliability in Parkinson’s monitoring.$Commercial implications: This paper guides production of more accurate gait analysis devices for clinical customers by revealing filtering-induced biases.
Authors
David Regev, Eran Beeri Bamani
Abstract
Video-based gait assessment often keeps only high-confidence pose observations, which can lower measurement error but change the movement assessed. We compared confidence selection with budget-matched random selection in a Parkinson-related cohort (n = 96) and an NF2-related replication cohort (n = 18). In the Parkinson-related cohort, confidence selection produced a small but consistent shift in the observed movement summary (+0.05 SD, 95% CI 0.03 to 0.07), while held-out Mini-BESTest error stayed flat (-0.04 points, -0.13 to 0.05); a policy preserving motion-group proportions reduced the shift, while NF2 showed no shift and its predictor did not beat a constant baseline. Because this shift mixes measurement error with selection shift, we decompose and simulate both on the cohort's window structure. Filtering lowers local error when error depends on confidence, but biases the endpoint once confidence tracks movement more than error. In the Parkinson-related data, confidence was only weakly tied to movement (rho = 0.14). There, filtering is harmless if well-tracked windows are unbiased. If instead systematic error only redistributes within participants, filtering can add up to 0.58 degrees. Unchanged clinical error from such small-cohort predictors therefore cannot show that quality filtering preserves the assessed movement.