Market forces shape ai safety investment in supply chains
Market Incentives for AI Safety Investment
Machine LearningComputer Science and Game Theory
Summary
AI systems can cause problems like spreading harmful content or misinformation, and keeping them safe costs money. The authors study how companies that make and use AI models decide how much to spend on safety. They find that companies competing with each other often spend more on safety than what would be best for the whole industry, but the main AI model provider might spend too little. They also explore how laws or rules could help encourage everyone to invest more in making AI safer.
What this means in practice
- •For ai product managers: Adjust investment strategies in AI safety measures based on competition effects within AI supply chains to optimize safety spending.
- •For policy makers: Design regulations that better align AI providers’ incentives with broader safety goals to prevent underinvestment upstream.
Authors
Nikita Tsoy
Abstract
Despite their practical significance, modern AI systems pose significant societal risks, including toxicity, malicious use, and misinformation. Their mitigation depends not only on technical feasibility but also on the incentives of firms that develop and deploy these systems. We show how profit maximization shapes safety investment in an AI supply chain with an upstream LLM provider and several competing downstream firms. In our game-theoretic model, we derive the market-driven level of safety investment and compare it with the levels that maximize market welfare and industry profit. We find that downstream competition generally induces downstream firms to invest more in safety than the industry-profit-optimal level. By contrast, the upstream provider may underinvest relative to these benchmarks because it is not fully compensated for the surplus from safety investments. Finally, we analyze how different regulatory interventions affect the participants' incentives and profits, and the resulting levels of model safety.