Robotic photography system understands instructions and captures pleasing photos
Robotic Photography System based on Vision-Language-Action Model
RoboticsArtificial Intelligence
Summary
Taking good photos with a robot is hard because the robot has to know what kind of picture is wanted, where to move, and how to compose the shot nicely. The authors created a new system called RoboPhoto-VLA that lets robots see, understand language instructions, plan their camera movements, and take pictures that look nice to people. They also made a new set of example photos and instructions to teach and test the system. When tested in the real world, their system took photos that matched people’s tastes better and worked well with new subjects and photo requests it hadn’t seen before.
What this means in practice
- •For robotics engineers: Create robots that autonomously capture photos matching complex verbal instructions and aesthetic standards in real-world settings.
- •For consumer camera manufacturers: Develop smart camera systems that can follow spoken or written photography directions to produce higher-quality images automatically.$Commercial implications: Enables sale of advanced autonomous photography devices that respond to user commands and preferences for better picture outcomes.
Authors
Song-Li Wu, Chubin Zhang, Wenbo Ding, Yansong Tang
Abstract
Modern robotic photography requires understanding photographic intent, planning camera motions, and producing aesthetically pleasing images in complex environments. We present RoboPhoto-VLA, a unified Vision-Language-Action framework that jointly models perception, reasoning, and camera control for autonomous robotic photography. RoboPhoto-VLA incorporates PhotoCoT for structured action planning and Aesthetic Direct Preference Optimization (ADPO) to align camera behaviors with human aesthetic preferences. To facilitate training and evaluation, we construct OpenPhotography, a robotic photography dataset with language-guided demonstrations. Extensive real-world experiments demonstrate that RoboPhoto-VLA produces photographs better aligned with human aesthetic preferences while generalizing effectively to unseen subjects, viewpoints, poses, and complex photographic instructions.