Papers for

human computer interface developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Person centered detection improves two hand and object interaction parsing

Single-Query Person-Centric Bimanual Hand-Object Interaction Detection

Abstract: Understanding person-level bi-manual interactions requires not only detecting hands, but also identifying which two hands belong to the same person and what each hand interacts with. Existing hand--object interaction methods are mostly hand-centric: they treat each hand as an independent instance, which can lead to ambiguous ownership in multi-person scenes. We propose a person-centric formulation in which a single query predicts a structured output for one person, including the human box, body pose, hand boxes and states, and interaction targets. We introduce part-aware deformable attention to allocate attention across human, hand, and pose-specific reference regions, enabling one query to capture the full person structure. We further unify detection and interaction reasoning with a hand-to-query relationship matrix, where each hand selects its interaction target from the detected query set plus a learnable off token, directly recovering the target's box and class without separate object regression. We build a COCO-based dataset with person-centric bi-manual interaction annotations and define structured metrics for evaluating hand states and complete hand--object tuples. Experiments with a transformer-based detector show that our formulation improves person-level bi-manual interaction parsing and provides an effective unified framework for joint detection, pose estimation, and hand reasoning.

Thu 10 SeptComputer Vision and Pattern Recognition
The gist
Understanding how people use both their hands to interact with objects is hard because it's tricky to know which hand belongs to which person and what each hand is doing. The authors created a new approach that looks at the whole person at once, including their body, hands, and what they might be holding or touching. Their method uses a special attention system to focus on important body parts and figures out hand interactions in one step. They also made a new dataset to test this and showed their method works better for recognizing these two-handed interactions.
Open 2609.12155v1