Billing methods affect language model cloud service cost and use
Who Pays for Thinking? Billing Hidden Reasoning Tokens in Model Cloud Contracts
Computer Science and Game Theory
Summary
When people pay for language model services, they might be charged not only for the answers they see but also for the model's hidden thinking steps used to get those answers. The authors studied how different ways of charging for this hidden computation influence the decisions of model providers and overall value for users and society. They found that charging per unit of computation can lead to extra computation that may or may not be worth the cost, while fixed prices limit how much computation is done. They tested this on real models and showed when each pricing method benefits users most.
What this means in practice
- •For cloud service providers: Design pricing models that balance computational costs and user satisfaction in language model offerings.
- •For language model developers: Adjust language model response generation strategies based on billing structures to optimize cost and performance trade-offs.
Authors
Chao Feng, Burkhard Stiller
Abstract
In token-priced language model services, users can see the prompts they submit and the answers they receive, but may also pay for internal reasoning and candidate answers that are not shown in the final response. This paper develops a three-party service model to study how charging for this intermediate computation shapes providers' decisions and social welfare. Users purchase answers from a model firm that pays an execution cloud, with either supplier controlling the computation budget. The model separates user billing from upstream payments and identifies when metered pricing encourages excessive generation, when fixed service prices restrict valuable computation, and how responsibility for execution costs changes these choices. A LiveCodeBench study measures answer quality, latency, token use, and resource consumption across five model deployments, with numerical billing comparisons for the three covered by archived tariffs. At equal average charges on the reference workload, the calibrated comparisons show that billing structure changes both service participation and the computation budgets suppliers select. For accepted services with different budgets, the measured outcomes define the user-value thresholds at which their welfare ranking reverses. Metered pricing yields higher welfare when the additional inference it induces creates net social value, whereas fixed service pricing is preferable when the costs of that computation outweigh its benefits.