Papers for

automated assembly line teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Visuo tactile model improves dexterous robot hand manipulation tasks

DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

Abstract: Dexterous manipulation depends on contact dynamics that are often only partially observable from vision. Recent World-Action Models (WAMs) couple predictive video world modeling with action generation, but remain largely vision-centric and therefore cannot directly model these contact dynamics. We present DexTacWAM, a visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a video diffusion world model for joint visuo-tactile world modeling. Across six contact-rich dexterous manipulation tasks on a 22-DoF bimanual platform, DexTacWAM achieves the highest score on every task, averaging 70.6 versus 38.0 for the strongest baseline. Ablations attribute the gain to modeling contact evolution as part of the predicted world state rather than tactile conditioning alone: removing tactile world modeling reduces the four-task mean from 74.7 to 26.6 while keeping the same tactile features and action expert. After four hours of tactile-encoder adaptation with a frozen pretrained vision VAE, our continual vision-to-touch learning extends the pretrained video model to touch using roughly 100 demonstrations per task without tactile midtraining, while retaining visual prediction quality within 0.5 dB of vision-only counterparts. The compressor retains 89.4% of pre-fusion contact recall while enabling 2.26x faster training and 1.29x faster inference. Together, these results show that pretrained video priors can be extended to distributed multi-finger contact dynamics in a data- and compute-efficient manner.

Mon 21 SeptRoboticsArtificial IntelligenceComputer Vision and Pattern Recognition
The gist
Manipulating objects with robotic hands is hard because cameras alone can't fully see how fingers touch and move on objects. The researchers created a model called DexTacWAM that uses both vision and touch sensors on each fingertip to better predict and control finger movements. This model learns from fewer examples and runs faster while performing much better on complex hand tasks than vision-only methods. Their approach extends existing video-based models to understand touch, which is important for delicate and precise robot hand operations.
Open 2609.24976v1