Forecasting tactile signals improves robotic handling of touch tasks
ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
Robotics
Summary
Robots often struggle to handle tasks that involve careful touching and feeling because they rely mostly on seeing things. The authors created a new method that helps robots predict how touch sensations will change over time to better guide their actions. This method combines what the robot sees, reads in instructions, and feels through touch sensors to plan movements more effectively. It was tested on real robots and showed much higher success in tricky tasks, even in poor lighting or cluttered spaces.
What this means in practice
- •For industrial robotics teams: Improve robotic arms to handle delicate assembly tasks by predicting future touch feedback for better grip and movement.
- •For warehouse automation engineers: Enhance robot performance in visually cluttered or low-light conditions by integrating predicted tactile information for more reliable item handling.
Authors
Zhengyu Tao, Xin Li, Xin Wang
Abstract
Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, yet their reliance on visual perception limits robustness in contact-rich environments, where critical physical interaction states may not be visually observable. Existing tactile-enhanced VLA methods improve physical grounding using observed tactile feedback, but most remain largely reactive rather than explicitly modeling how contact may evolve. Therefore, we propose ForeTac-VLA, a forecasting-based tactile-vision-language fusion model that predicts future tactile states to guide action generation. Specifically, ForeTac-VLA encodes recent tactile observations into temporal representations and integrates them with vision-language features through bidirectional cross-attention. Further, a transformer-based forecasting module predicts multi-step future tactile states, enabling the model to reason jointly over observed and anticipated contact. Finally, the fused multimodal representations and predicted future tactile states are fed into the VLA backbone to condition action generation. To stabilize training, a ground-truth-to-prediction curriculum is employed when early forecasts are unreliable. Across four real-world contact-rich manipulation tasks, ForeTac-VLA achieves an average success rate of 95%, outperforming the fine-tuned VLA model by 36.25 percentage points and state-of-the-art tactile-enhanced VLA baselines by over 22 percentage points. ForeTac-VLA also maintains strong performance under low-illumination and visually cluttered conditions. Video demonstrations can be found on https://foretac-vla.github.io/