Papers for

home automation developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

ARSTAG creates robot training data from images and instructions

ARSTAG: An Agentic Real2Sim2Real System for Task-Specific Robot Data Generation

Abstract: Adapting visuomotor policies to new manipulation tasks often requires substantial manual engineering or teleoperated data collection. Simulation can provide task-specific data at scale, but constructing the scene, designing expert behavior, and configuring data generation still require significant per-task effort. We present ARSTAG, an agentic Real2Sim2Real system that turns a single RGB image and a natural-language instruction directly into robot policy-learning data. A hierarchy of language agents constructs a task-scoped simulation scene, generates robot-feasible demonstrations, and expands the training distribution through task-consistent randomization, while a coordinator agent manages cross-stage feedback and recovery. Across seven manipulation tasks spanning grasping, placement, and stacking, the ARSTAG-generated demonstrations enable sim-to-real transfer of three visuomotor policy architectures to a dual-arm robot, with pi0.5 achieving an average real-world success rate of 74.6%. Ablations show that task-consistent randomization substantially improves robustness, and policy performance increases with generated dataset size. Project webpage: https://boweili666.github.io/ARSTAG/.

Mon 21 SeptRobotics
The gist
Robots often need lots of hand-coded setup or human guidance to learn new tasks involving objects. The authors present ARSTAG, a system that takes a single picture and a simple language instruction to automatically create training data for robots. It builds a virtual scene, generates robot actions, and creates varied examples to help robots learn better. Their tests show that robots trained with ARSTAG's data can perform different manipulation tasks with good success rates in real life.
Open 2609.24563v1