Relational scene graphs improve autonomous driving prediction accuracy
Relationally Grounded Latent World Models for Autonomous Driving
Computer Vision and Pattern RecognitionRobotics
Summary
Predicting how cars and pedestrians move is important for self-driving cars. The authors explore using 'scene graphs'—which show how things in traffic relate to each other—to teach a computer better. They train their model to recognize these relationships during learning but do not rely on them later. This method reduced prediction errors and collisions in tests, showing understanding relationships helps improve driving safety models.
What this means in practice
- •For autonomous vehicle developers: Enhance trajectory prediction models by incorporating relational scene graph supervision to reduce collision rates without extra runtime cost.
- •For robotics software engineers: Improve latent state representations in robot navigation systems by training with explicit relational semantic targets derived from scene graphs.
Tested on one dataset.
Authors
Fabian Schmidt, Markus Enzweiler, Abhinav Valada
Abstract
Latent world models learn predictive representations for autonomous driving, but the relational semantics these states preserve often remain implicit. We investigate whether traffic scene graphs can serve as privileged semantic supervision for latent world representations. Building on LAW, we construct actor-centric scene graphs from nuScenes 3D annotations, encode their serialized relational structure using a frozen text embedding model, and align the visual latent representations with this semantic target during training. We remove the supervision branch at inference, so it requires neither scene graphs nor 3D annotations and adds no test-time computation. On nuScenes, our method reduces average trajectory L2 error from 0.661 to 0.622 (5.9%) and collision rate from 0.456 to 0.217 (52.4%) relative to our retrained LAW baseline. It also outperforms an unstructured caption-style semantic target, supporting the benefit of explicit relational structure for latent world-model representation learning.