Evaluation choices affect measured benefits of cooperative vehicle perception
How Evaluation Choices Change the Measured Benefit of Cooperative Perception: Evidence from Three V2X Benchmarks
Computational Engineering, Finance, and Science
Summary
Cooperative perception helps connected vehicles and roadside devices share sensor data to improve automated driving. This paper shows that the way researchers measure accuracy can greatly change how much benefit cooperative perception seems to provide. The authors found that in simulated and real-world tests, tweaking evaluation details like data splits or class definitions can reduce or inflate the perceived improvements. Therefore, claims about the advantages of cooperation need to specify exactly how the measurements were made.
What this means in practice
- •For automotive engineers: Design testing protocols that consider partner availability and pose errors to accurately assess cooperative perception benefits in vehicle systems.
- •For transportation planners: Make informed decisions on roadside infrastructure deployment by understanding how measurement choices impact perceived gains from vehicle cooperation.
Authors
Pincan Zhao, Yili Tang, Xinrui Zhang
Abstract
Cooperative perception, in which connected vehicles and roadside infrastructure share sensor information, is a candidate enabler of automated mobility, and benchmark accuracy is the evidence cited when roadside deployment is considered. This paper audits that evidence base across one simulated and two real-world vehicle-to-everything (V2X) benchmarks. In simulation, two widely studied robustness axes leave almost no recoverable headroom: an infrastructure-anchored pose correction returns about one accuracy point at every error level, and corrupting a partner costs 0.7 points. On real data, measurement choices govern the conclusion. An apparent seventeen-fold advantage of infrastructure in partner-poor frames falls below four-fold once the split is broadened, and an ad-hoc class definition measures a far smaller benefit than the official protocol reports. Cooperation is worth 7 to 15 accuracy points, yet 27% of frames on one split offer no partner and almost none do on another, so every regime-conditioned claim must name its split.