NEO: NeRF It Once, Edit It Many Times for Continuous Object Manipulation
2026-07-27 • Robotics
Robotics
AI summaryⓘ
The authors created NEO, a system that lets robots change 3D scenes using language instructions. They combined methods to remove objects from scenes and taught a model to predict how the scene should look after changes. They also made a new dataset to test how well these editing methods work for robots. Their approach produces better and more realistic changes than earlier methods, helping robots plan actions more accurately.
NeRFNeural Radiance Fieldsrobotic manipulationlanguage-guided editinginpaintingknowledge distillationteacher-student modelscene editing benchmarkobject removal
Authors
Mikołaj Zieliński, David Hall, Dominik Belter, Peyman Moghadam
Abstract
In this paper, we present NEO, a unified framework providing language-guided NeRF editing for robotic manipulation. Our paper introduces (i) a language-guided object removal that combines neural field resampling with multiview-consistent progressive inpainting, (ii) a direct NeRF weight editing method utilizing knowledge distillation, composing original and edited NeRFs via a teacher-student model, enabling coherent modeling of future scene states before a robot executes an action, and (iii) the first benchmark (NEO-Dataset) for quantitatively evaluating NeRF scene editing methods suitable for robot manipulation. We show that our approach outperforms state-of-the-art baselines in scene editing tasks, including object removal and pick-and-place robotic experiments, yielding visually coherent and geometrically consistent edits that reduce artifacts commonly introduced by prior methods.