Robot learns best camera views to inspect assembly parts automatically
INSPECT: Learning Robot View Selection from Assistant Use
RoboticsComputer Vision and Pattern Recognition
Summary
Robots inspecting assembled parts need to see the right angles to check if everything is present and correctly installed. The authors developed INSPECT, a system that learns which views a robot should select by studying how a smart-glasses assistant helps humans during assembly tasks. INSPECT uses special techniques to understand which views provide the most useful evidence without needing example labels from the target robot. Tests with physical gearboxes and other equipment showed INSPECT improves the robot’s ability to verify parts and helps it make better inspection decisions.
What this means in practice
- •For industrial robot integrators: Enable robots inspecting assembled machinery to autonomously select camera views that best verify part presence and correctness without manual tuning.$Commercial implications: This paper enables building commercial robotic inspection products that reduce manual setup by learning view preferences from human-assistant data.
- •For automation system engineers: Improve quality-control systems by integrating learned camera view policies that enhance defect detection accuracy during automated assembly checks.
Authors
Di Wen, Kailun Yang, Wenhao Guo, Yitian Shi, Junwei Zheng, Yufan Chen, Ruiping Liu, Jiale Wei, Rania Rayyes, Kunyu Peng
Abstract
Robots inspecting an assembly must determine which parts are present and whether they are correctly installed. During egocentric assembly assistance, head motion and workpiece handling reveal evidence for these checks, while spoken state confirmations link observations to procedural outcomes. We introduce INSPECT, which learns robot view preferences from records of a smart-glasses assistant that answers part queries and provides next-step guidance. Presence-Invariant TwinSwap (PI-TwinSwap) calibrates object evidence through paired identity interventions. Claim-indexed supervision separates evidence requirements from camera-reproducible observation changes. Object-centered calibration adapts relative view preferences to robot poses, while clause-level screening checks predicted evidence. The robot selects views using only its current observation and known poses, without candidate images. Evaluation uses annotated assistant-video replay to simulate state feedback, without target-domain view labels for policy training. On images of physical gearbox assemblies, INSPECT achieves the highest view utility among the compared non-oracle policies and raises human-rated full verifiability from 34.8% to 41.7% compared with keeping the current view. On commercial angle-grinder recordings in IMPACT, the transferred relative-view selector increases the correct decision rate from 50.6% to 54.3% with a frozen perception head. The source code is available at https://github.com/Kratos-Wen/INSPECT.