Fine-grained robot control improves task success with language input
Task-Specialized Language-Conditioned Models for Fine-Grained Robotic Manipulation
Robotics
Summary
Fine control of robot hands is hard because it requires understanding instructions, seeing details well, and touching objects carefully. The authors combined five separate AI models that use language and vision together to pick good actions for different tasks. They improved training data by checking robot grasps and fixing mistakes, and used smart techniques when running the robot to reduce errors. Their system ranked third in an international competition that tests these skills. They also studied how the models behave differently for each task.
What this means in practice
- •For robotics engineers: Control robots in manufacturing to follow spoken or written instructions for precise assembly tasks using improved language-guided models.
- •For automation system integrators: Incorporate task-specialized robot models into complex automation lines to enhance performance on delicate manipulation steps.
Authors
Chanho Kim, Soyun Choi
Abstract
Fine-grained robotic manipulation requires instruction grounding, robust perception, and precise contact control. We describe our submission to the MetaFine Open Competition at CoRL 2026. Five independently trained models use a common architecture with frozen DINOv2 and CLIP encoders, trainable language-conditioned feature modulation, and task-specific action heads. We improve demonstrations through contact-aware grasp filtering, instruction-matched data, and expert correction of insertion failures. At inference, task-dependent strategies include photometric test-time augmentation, stochastic action averaging, and uncertainty-triggered replanning. Our submission achieves an official overall score of 0.914 and ranks third among 15 teams. We analyze the models and their task-specific behavior.