Summary
Crash severity models help decide how to respond to accidents, but it’s unclear how reliable individual predictions are because the data and conditions vary a lot. The authors developed a special method that can wrap around any severity prediction model to provide solid guarantees about its predictions without assuming a specific data distribution. Their method accounts for errors in reported crash severity, works across different regions and times, and controls risks based on severity levels. They tested this method on millions of Texas crash records over decades, showing it sets a reliable benchmark no model can beat and provides clearer, trustworthy risk assessments.
What this means in practice
- •For traffic safety analysts: Provide crash-severity prediction tools with reliable, quantifiable guarantees that remain valid across regions and years, improving safety prioritization.
- •For emergency dispatch teams: Ensure severity predictions used in dispatching have trustworthy risk bounds despite varying reporting errors and changing conditions over time.
Abstract
Crash-severity models inform screening, dispatch and site prioritization, yet are deployed without a finite-sample statement of what one prediction means. Off-the-shelf guarantees fail here, because the features that make crash severity distinctive defeat them: the KABCO outcome is ordinal, the recorded label is a field assessment agreeing with medical severity about half the time, erring in a structured way, and deployment crosses jurisdictions and years calibration never saw. We develop a certification layer that wraps any severity model unmodified, with distribution-free guarantees using this structure: contiguous ordinal sets that read as "B or worse"; per-class validity for any pre-declared partition, with an oracle efficiency characterization; transfer of coverage to unobserved true severity through a declared reporting band, with a worst-case sharpness result; a one-sided certificate under deployment shift; and severity-weighted risk control. The guarantees compose with an attributable slack budget. The same analysis bounds what certification can achieve. A certified set's informativeness is governed by a functional of the true law that no base model can evade and that cannot be lower-bounded distribution-free; given a declared misreporting channel identified from record-linkage data, a nonvacuous lower bound on that floor becomes computable. On 5.2 million Texas records across seven base models spanning four decades, the layer attaches identical validity and certifies, on the vulnerable road users, a model-independent floor on set width that no base model beats, separating it from a remainder that stays bounded but distribution-free unidentifiable. The framework is released as an open-source package with theorem-level tests.