Strategic calibration improves truthful confidence reporting among AI agents

Strategic Calibration in LLM Multi-Agent Systems: Audited Pivot Scoring for Truthful Elicitation

Computer Science and Game TheoryArtificial Intelligence

Summary

As large AI systems use multiple agents working together, how confident each agent says it is can change what tasks get priority and how decisions are made. The authors show that agents might claim to be more confident strategically, not truthfully. They introduce a new method called Audited Pivot Scoring Mechanism to encourage agents to report their true confidence by balancing rewards and penalties using audits and proper scoring. This method helps make AI collaborations more honest and reliable.

What this means in practice

  • For ai system engineers: Design multi-agent AI systems where agents truthfully report confidence to improve task allocation reliability.
  • For platform operators: Implement incentive-aware scoring mechanisms to reduce misreporting and optimize resource use in AI-assisted decision systems.

Authors

Jinhao Zhou, Zhaohan Zhang, Xijian Xu, Jun Wu

Abstract

In large language model (LLM)-based multi-agent systems, reported confidence increasingly determines the allocation of scarce downstream attention, computation, and decision influence. This makes reporting strategic: an agent may overstate the relevance of its contribution even when its underlying belief is well calibrated. Existing calibration methods largely assume passive reporting and therefore do not address strategic incentives created when confidence determines downstream allocation. We formulate strategic calibration as a reporting-layer incentive problem and propose the Audited Pivot Scoring Mechanism (APSM). APSM offsets platform-controlled allocation benefits, uses inverse-propensity-weighted sparse audits, charges the Clarke pivot externality, and applies a strictly proper score. Under the stated model assumptions, truthful reporting is the unique expected-utility best response. We also characterize approximate execution and the audit-variance trade-off. Calculator-free experiments show that LLM agents inflate confidence under Top-$K$ incentives. With a shared agent-local utility interface, LLM agents reduce distortion under both Compensated Proper Scoring (CPS) and APSM. CPS achieves higher exact allocation match, whereas APSM yields lower Brier loss and numerically fewer cutoff crossings. Experiments further show that APSM enlarges the utility margin against allocation-changing deviations by $1.93\times$ and improves robustness under approximate execution. These results support incentive-aware elicitation as a distinct layer for reliable multi-agent coordination.