World model planning shows uneven effects of sensing disturbances
Beyond Task Success: Stage-Wise Reliability of World Model Planning under Sensing Degradation
RoboticsMachine Learning
Summary
When a robot or AI uses its internal model to decide what to do, it relies on sensing inputs that go through several steps before action. The authors show that if the sensing gets worse in some way, the problem can affect different steps—like how the robot understands the world, predicts the future, picks actions, or actually succeeds—in different ways. Sometimes big issues early on fade out later, and sometimes small early changes cause big problems at the end. Knowing exactly where the sensing problems matter most can help improve these AI systems.
What this means in practice
- •For robotics engineers: Diagnose which stages in the robot’s decision pipeline suffer most from sensor noise to better target hardware or software fixes.
- •For autonomous vehicle developers: Improve sensing and model verification prioritization by identifying where environmental sensing errors persist or fade within decision-making systems.
Authors
Geonmyeong Lee, Byoung-Tak Zhang
Abstract
In world model planning, sensing inputs pass through an encoder and predictor before affecting planner decisions, so final task success alone cannot reveal where sensing disturbances attenuate or persist in the pipeline. We apply 10 visual and temporal sensing degradations to a world model planner and track their effects across representation, future prediction, planner preference, and physical outcome using paired evaluation on the same 50 tasks. The relative impact of degradations was not preserved across stages: large representation shifts could attenuate downstream, while smaller initial shifts could persist to the outcome, and internal-response ordering did not directly match physical-outcome ordering. Temporal degradations also showed distinct patterns: even with similar overall changes in observation history, responses differed substantially with the location of corrupted information and the planner's actual exposure. This non-uniform stage-wise response was also observed in secondary evaluations with another manipulation task and a different world model. Stage-wise diagnosis can therefore identify where sensing disturbances attenuate or persist and help prioritize subsequent model verification and sensing mitigation.