Portable robot hand device captures detailed finger and contact data
DEX-PHD: A Dexterous Portable Handheld Device for Multimodal Data Collection
Robotics
Summary
Collecting data about how hands move to manipulate objects is either very accurate but bulky, or easy but less detailed. The authors created a handheld robot hand device that captures precise finger movements, touch data, and video while being portable. People control its fingers directly and it works in many different places. They showed it works well for tricky tasks needing careful finger use and can replay the motions on a larger robot.
What this means in practice
- •For robotics engineers: Collect detailed hand manipulation data in diverse environments without bulky equipment for developing dexterous robot control.
- •For robot arm manufacturers: Create more precise multi-fingered robot hand controllers by using detailed datasets from portable devices to improve task performance.$Commercial implications: Enables sale of advanced robot hand controllers with improved dexterity for industrial and service robots through richer data collection.
Authors
Jooyoung Hong, Noboru Myers, Obin Kwon, Yixian Yang, Joohyung Kim
Abstract
Existing manipulation data collection methods typically offer either accurate robot-state acquisition through kinematic-chain-based teleoperation or convenient, portable operation through vision-based approaches. We present DEX-PHD, a portable handheld device that combines these advantages for dexterous manipulation by integrating a dexterous robot hand, continuous per-finger control, and multimodal logging. The system records proprioceptive information, fingertip contact data, and wrist-camera video, while the operator directly controls the robot hand through joint-level mapping. Its untethered design enables data collection across diverse environments. We demonstrate DEX-PHD on contact-rich tasks requiring coordinated multi-finger manipulation and deploy the collected demonstrations through open-loop replay on a physical humanoid robot. The results show that open-loop replay preserves the main temporal patterns of finger motion and task-relevant contact. Therefore, DEX-PHD provides a practical platform for portable, multimodal dexterous manipulation data collection.