Contracts protect AI service users from hidden quality drops
Anytime Contracts for AI Services
Computer Science and Game TheoryMachine Learning
Summary
AI services sometimes lower their quality without telling users to save costs, which is hard to spot because results can naturally vary. The authors study how to set up payment agreements that encourage AI providers to keep quality high, even though users can stop using the service anytime. They use math from contract theory and create new kinds of contracts that keep working no matter when users leave. These contracts are based on statistical tests that can catch quality drops quickly. Their methods were tested on both simulated data and real data from large language models.
What this means in practice
- •For ai platform operators: Design subscription plans that fairly pay for real AI quality and detect silent provider downgrades at any time.
- •For cloud service architects: Develop API billing systems that adjust charges dynamically based on ongoing detection of model performance changes.
Authors
Boris Velasevic, Eden Saig, Eric Mazumdar
Abstract
AI services are prone to moral hazard, as model providers can silently route requests to cheaper, lower-quality models, and the inherent stochasticity of model outputs makes this degradation hard to detect. Contract theory addresses moral hazard by designing payment rules that reward high-quality outputs and prevent strategic overcharging. However, existing frameworks assume interactions of fixed duration, whereas AI services are typically consumed through subscriptions and pay-as-you-go APIs that users can terminate at any time based on observed performance. We show that this flexibility can break standard contracts, opening them up to abuse by either the providers or users. To address this, we extend contract theory to account for unknown exit rules. We formalize the problem as a Principal-Agent game, and show that budget-optimal contracts in this setting are proportional to anytime-valid hypothesis tests based on likelihood-ratio e-values. We give an efficient algorithm for computing anytime contracts and show how betting e-processes can be used to approximate them when the models are unknown. Finally, we evaluate our findings on synthetic data and data from deployed LLMs.