A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems
2026-04-01 • Robotics
RoboticsArtificial IntelligenceComputer Vision and Pattern Recognition
AI summaryⓘ
The authors created a software tool that helps robots use a vision and language model called Florence-2. This tool lets robots understand images and text in several ways, like recognizing objects or reading text, through a system called ROS 2. They made sure it works well on common computers and showed how fast it runs on different graphics cards. Their work focuses on making it easier for developers to add advanced vision-language functions into robots.
Foundation vision-language modelsFlorence-2ROS 2Robotics middlewareOptical character recognitionOpen-vocabulary detectionDockerGPU throughputRobot software stack
Authors
J. E. Domínguez-Vidal
Abstract
Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: https://github.com/JEDominguezVidal/florence2_ros2_wrapper