Papers for

interactive hardware developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Robot drawing benchmark reveals challenges in language guided actions

BURGER: Benchmarking and Understanding Robotic Generalization via Embodied Drawing

Abstract: How effectively can vision-language-action (VLA) models learn diverse language-conditioned behaviors as training data scales? Investigating this question requires large and varied robot datasets, whose collection on physical robots remains costly. We introduce BURGER, a language-conditioned robotic drawing benchmark with a scalable, prompt-driven pipeline for generating continuous drawing trajectories, both in simulation and the real world. In BURGER, an agent starts from the same blank canvas and controls a robot arm to draw the object or spatial composition specified by language, while each instruction allows an open-ended variety of valid drawings and stroke orders. Using 28,200 simulated demonstrations spanning 282 drawing tasks, we evaluate five representative VLA models across data scales, language grounding, and multimodal behavior. BURGER supports fine-grained analysis of language grounding, spatial composition, and action generation, revealing distinct failure patterns across VLA models. Our experiments suggest that some VLA models can benefit from greater task diversity and demonstration scale, capture multimodal drawing behaviors, and generalize to unseen compositions of familiar objects. Finally, we deploy a trained policy on a Franka Panda arm, demonstrating the ability to draw reactively under human intervention. Project homepage and code can be found at https://siyingzhao.github.io/BURGER/.

Thu 29 OctRoboticsArtificial Intelligence
The gist
Teaching robots to understand language and draw pictures based on those instructions is hard, especially because real-world training is expensive. The authors created BURGER, a robotic drawing test that uses language to guide a robot arm in drawing shapes both in simulation and real life. They tested multiple AI models to see how well they learn to follow instructions and create drawings. Their results show differences in how models understand language and actions, and some can handle new drawing tasks by combining known elements. They even demonstrated a robot drawing while reacting to humans in real time.
Open → 2610.40532v1

Multicamera dataset enables low cost screen detection and pointing estimation

DABACO: A Multi-Camera Dataset and Benchmark for Screen Localization and Pointing Estimation

Abstract: Screen localization and pointing estimation are key to low-cost interactive devices. Yet developing and evaluating these algorithms requires realistic data: synthetic captures cannot fully reproduce the optical distortion, rolling shutter, motion blur, and display processing of a physical acquisition, and most existing datasets provide static images rather than the video needed to assess continuous pointing. In this paper, we introduce the DABACO Dataset. Developed within the DABACO (Dispositivo Apuntador de BAjo COste, or Low-Cost Pointing Device) project, this dataset supports the development and evaluation of screen detection and camera-based pointing algorithms for embedded systems. It comprises video sequences captured with multiple low-cost camera sensors, including monochrome global-shutter and color rolling-shutter modules, on embedded platforms such as the Raspberry Pi 4B and ESP32-S3. We present a systematic annotation pipeline combining temporary visual watermarking, optical-flow tracking, manual verification, and marker removal. Both the original marked captures and the marker-free images, together with explicit corner annotations and modification masks, are released to support auditing and the study of potential reconstruction bias. In addition, we release an open-source evaluation toolkit with two reference baselines: a classical screen-detection pipeline based on edge and contour geometry, and a closed-vocabulary, off-the-shelf YOLOv8 segmentation model evaluated without dataset-specific training. Both are evaluated on a general-purpose computer using Intersection over Union, corner error, and pointing error, establishing initial reference results for future embedded implementations. Overall, DABACO addresses a gap in existing resources and is intended to support the development and evaluation of new low-cost screen-localization and pointing systems.

Fri 2 OctComputer Vision and Pattern RecognitionMultimedia
The gist
Detecting screens and where someone is pointing at them using cameras is important for making affordable interactive devices. The paper introduces the DABACO Dataset, which contains video clips from multiple inexpensive cameras set up on small computers. This dataset helps researchers and developers test and improve methods for finding screens and estimating pointing gestures, using real video rather than just pictures or synthetic data. The authors also provide tools to evaluate how well these methods work, along with some example approaches as a starting point.
Open → 2610.03928v1