Active evidence acquisition improves decision making under cost constraints
ActiveEvidence: Decision-Focused Evidence Graphs with Verifiable Acquisition Gaps
Artificial IntelligenceComputer Vision and Pattern RecognitionMachine Learning
Summary
Deciding when and what visual evidence to gather can be costly and complex for AI agents. The authors present a framework that smartly chooses which evidence to acquire and when to stop gathering more, ensuring decisions stay reliable without unnecessary expenses. They achieve this by using a special graph to model evidence interactions and certify that no better choice exists beyond a certain tolerance. Their method was tested on many cases, showing it reduces work while keeping decision quality high.
What this means in practice
- •For autonomous vehicle teams: Optimize sensor queries to decide when to acquire costly visual data for accurate driving decisions under uncertain conditions.
- •For security monitoring operators: Reduce expensive video analysis by certifying when additional camera views do not improve threat detection decisions.
Authors
Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Peng Zhang, Daren Zha, Jun Xiao
Abstract
A multimodal agent must decide whether visual evidence is worth acquiring, which candidate or resolution to query, and when further interaction can no longer change its decision. We study this problem under a strict free-metadata/paid-observation boundary, stochastic vector costs, and a frozen execution trace, and treat it as decision-certifiable active evidence acquisition. A Counterfactual Evidence Interaction Graph (CEIG) represents unary evidence value, signed second-order interaction, and directed conditional harm while maintaining explicit score enclosures for unresolved interactions; a decision-weighted enclosure objective tightens the bounds that can change action identity. Gap-directed refinement evaluates only interactions that can displace the incumbent acquisition and stops once an $ε$-decision certificate shows that unresolved terms cannot change the selected action beyond a declared tolerance. The certificate is model-relative: it verifies policy-consistent optimality under the pinned learned score, and semantic quality is measured separately through post-freeze labels. Paired continuation rollouts learn the downstream value of paid previews, an online operational cost filter is separated from independent finite-policy-family trajectory certification, and an acquisition monitor with an independently implemented verifier binds every certificate to the frozen trace. On 640 cases split into 384/128/128 training, validation, and test cases, the controller reaches task score 0.842, grounding F1 0.816, evidence recall 0.873, 1.74 accepted calls, 286.5K visual tokens, and 742ms p95 latency. Dense replay of issued certificates shows 0.992 action agreement and a 0.988 certificate rate while evaluating 19.6 instead of 191.4 expensive interaction terms per decision.