Papers for

risk assessment analysts

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Threshold aware routing cuts costly simulator calls by up to 75 percent

Threshold-Aware Conformal Routing

Abstract: High-fidelity simulations are essential to scientific and engineering design, but can be expensive to run repeatedly. Learned surrogates offer a faster alternative, yet their higher errors may alter downstream decisions. This accuracy-speed tradeoff creates a need to determine whether a surrogate can be used or the full simulator remains necessary. We study decisions determined by whether a scalar quantity of interest lies above or below a fixed threshold. For each input, we use the surrogate when its conformal interval lies entirely on one side of the threshold and route the input to simulation when the interval intersects it. Standard conformal prediction constructs intervals without reference to the downstream decision threshold: even a narrow interval near the threshold can cross it and trigger simulation, whereas a wider interval farther away can remain entirely on one side and require no simulation. We introduce Threshold-Aware Conformal Routing (TACR), which learns an input-dependent scale using a threshold-aware objective that concentrates interval tightness near the decision boundary. Exact split-conformal calibration on held-out data preserves distribution-free marginal coverage, which also upper-bounds the probability of an incorrect threshold decision that is not routed. Across various scientific and engineering datasets, TACR reduces simulator deferrals by 14-75% relative to standard conformal prediction at the same coverage target. Against a variant without threshold-local weighting but with similar predictor accuracy, TACR further reduces deferrals by 10-24% on four datasets. These results show that optimizing interval allocation for routing can reduce simulator calls without weakening the standard conformal guarantee.

Thu 1 OctMachine LearningComputational Engineering, Finance, and Science
The gist
Running detailed simulations can be slow and expensive, so people sometimes use faster approximate models called surrogates. But these surrogates can make mistakes that affect decisions about whether a measurement is above or below an important cutoff. The authors introduce a method called Threshold-Aware Conformal Routing (TACR) that decides when to trust the surrogate and when to run the full simulation, based on how sure the surrogate is about being above or below the cutoff. TACR focuses on making prediction intervals tighter near the threshold, improving decision accuracy and reducing the number of costly simulator runs.
Open → 2610.02487v1