Papers for

farm equipment builders

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Vision language action models improve robot grasping in agriculture

Robot Learning with Vision-Language-Action Models for Agricultural Manipulation

Abstract: Robotic manipulation in agricultural environments remains challenging due to variations in plant geometry, occlusion, and camera viewpoints. Pretrained vision-language-action (VLA) and generalist robot policies offer a promising approach to addressing these challenges by learning manipulation behaviors from multimodal observations and language instructions. However, adapting these policies to agricultural tasks requires domain-specific demonstrations, which are costly and time-consuming to collect on physical robots. In this work, we develop a simulation-to-real robot-learning framework for agricultural reaching and grasping that integrates task-specific demonstration collection and pretrained policy adaptation. A MuJoCo-based agricultural environment is developed with diverse tomato plant structures to support demonstration collection and controlled evaluation. Expert demonstrations are collected through joystick teleoperation, with a low-level controller used to execute robot actions. Two pretrained generalist robot policies, Octo and $π_{0.5}$, are adapted using simulated demonstrations, followed by further fine-tuning on a physical Franka Research 3 manipulator using real-robot demonstrations. The adapted policies are evaluated under in-distribution, viewpoint/spatial-shift, and out-of-distribution conditions in simulation, followed by physical-robot experiments. The influence of proprioceptive information on policy performance is also investigated. Results show that $π_{0.5}$ achieves higher task-performance scores than Octo and exhibits less degradation under distribution shifts. These findings demonstrate the potential of integrating pretrained generalist policies with task-specific demonstration learning for agricultural manipulation.

Mon 19 OctRobotics
The gist
Robots find it hard to pick fruits and vegetables because plants can look very different and cameras can see things from many angles. The authors worked on teaching robots to understand what to do using both images and language. They used a computer simulation to train robots on how to reach and grasp tomatoes and then tested their learning on a real robot arm. They found one robot policy worked better and stayed more reliable when conditions changed. This shows combining learning from language, vision, and actions helps robots handle tricky farm tasks.
Open → 2610.26717v1