TriO predicts driving space and obstacles using three sensor types
TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Understanding the environment around a vehicle is hard because it needs to identify many different objects and obstacles with different sensors. The authors created TriO, a model that combines camera, LiDAR, and RADAR data to learn without needing extra human labeling. This model can predict where objects are, how they move over time, and tell apart the road from obstacles. It does this better than previous methods, even for unusual or rare objects.
What this means in practice
- •For autonomous vehicle engineers: Use TriO's multi-sensor prediction to improve environment perception and obstacle handling without manual labeling.
- •For robotics navigation teams: Incorporate TriO’s occupancy and flow predictions in robot path planning for safer movement in varied environments.
Authors
Quinlan Sykora, Sourav Biswas, Christopher Diehl, Andrew Cunningham, Thomas Gilles, Raquel Urtasun
Abstract
We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to prior work, TriO utilizes three distinct sensor modalities (camera, LiDAR, and RADAR) as both inputs and sources of self-supervision, eliminating the need for additional human annotations. Thanks to its novel supervision, the model is able to segment any occupancy from the drivable surface, overcoming the limitations of existing open-set methods in handling long-tail objects. TriO achieves state-of-the-art results in multiple 3D and 4D tasks, including occupancy, flow, and LiDAR prediction, as well as zero-shot road obstacle segmentation across multiple datasets such as Argoverse 2, and Spotting the Unexpected.