World model planning shows varied effects of sensing damage at different steps
Beyond Task Success: Stage-Wise Reliability of World Model Planning under Sensing Degradation
RoboticsMachine Learning
Summary
When computers use world models to plan actions, they rely on sensor inputs that go through several steps: encoding, prediction, planning, and then the final outcome. The authors found that problems caused by sensor damage don’t always affect every step equally—big problems early on can sometimes get smaller, while small early problems might still cause trouble later. They also saw that where and when the sensor data is damaged changes how planning is affected, especially with timing issues. Their approach helps spot exactly where and how sensor problems cause trouble, which could help improve these systems.
world modelplanningsensor degradationencoderpredictorrepresentationfuture predictionplanner preferencetemporal degradationmodel verification
Authors
Geonmyeong Lee, Byoung-Tak Zhang
Abstract
In world model planning, sensing inputs pass through an encoder and predictor before affecting planner decisions, so final task success alone cannot reveal where sensing disturbances attenuate or persist in the pipeline. We apply 10 visual and temporal sensing degradations to a world model planner and track their effects across representation, future prediction, planner preference, and physical outcome using paired evaluation on the same 50 tasks. The relative impact of degradations was not preserved across stages: large representation shifts could attenuate downstream, while smaller initial shifts could persist to the outcome, and internal-response ordering did not directly match physical-outcome ordering. Temporal degradations also showed distinct patterns: even with similar overall changes in observation history, responses differed substantially with the location of corrupted information and the planner's actual exposure. This non-uniform stage-wise response was also observed in secondary evaluations with another manipulation task and a different world model. Stage-wise diagnosis can therefore identify where sensing disturbances attenuate or persist and help prioritize subsequent model verification and sensing mitigation.