Detecting Pen-In-Air States from Video: A Proof-of-Concept Toward Complementary Handwriting Analysis
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how to detect when a pen is lifted off the paper during handwriting using regular video from above, instead of relying on special tablets that only sense close to the surface. They used a computer program to track the pen tip in video frames and then analyzed the movement to decide if the pen was touching the paper or not. Their method was tested on labeled handwriting videos and was good at spotting pen lifts, especially focusing on not missing any. This approach could add a low-cost way to understand handwriting motions without needing expensive equipment.
Pen-Up detectionhandwriting analysisdigitizing tabletsYOLO object detectionkinematic featuresmachine learning classificationLeave-One-Video-Out protocoldysgraphia assessmentvideo-based tracking
Authors
Lauren Sismeiro, Remy Plastre, Binbin Xu, Frederic Puyjarinet, Gerard Dray
Abstract
Dynamic aspects of handwriting are critical for assessing developmental disorders such as dysgraphia and are typically captured using digitizing tablets. However, tablet-based sensing restricts analysis of Pen-Up behavior to a short proximity range above the writing surface, potentially missing high-lift in-air movements. As a proof of concept, we investigate whether top-view video can provide a complementary source of information for inferring pen-contact states without relying on tablet proximity sensing. We propose an interpretable hybrid pipeline combining pen-tip tracking using a YOLO-based detector with kinematic feature extraction and machine learning classification. A pilot dataset of diverse handwriting videos was manually annotated at the frame level and evaluation used a Leave-One-Video-Out (LOVO) protocol. The method achieved reliable event-level detection of Pen-Up segments, with an F_2 score up to 0.805, consistent with the emphasis on recall in a screening-oriented setting. These results support the feasibility of video-based Pen-Up detection as a low-cost and non-intrusive complement to digitizing tablets, and provide a foundation for future large-scale studies.