Llm advice changes when same risks are framed as gains or losses
Same Probabilities, Different Advice: Gain-Loss Framing Effects in High-Stakes LLM Decision Support
Artificial IntelligenceComputation and Language
Summary
LLMs (large language models) are often used to help with important decisions, but how they give advice can change depending on how probabilities are described. This paper shows that when the same chance is expressed either as a possible gain or a possible loss, the LLMs give different recommendations about one-third of the time. These changes do not usually reverse the advice completely but affect how certain or uncertain the models sound. The authors also found that simplifying how probabilities are represented can make LLM advice more consistent in high-stakes situations.
What this means in practice
- •For healthcare providers: Improve clarity and consistency of LLM decision support in medical risk communication by adjusting probability expressions to reduce framing bias.
- •For financial advisors: Develop tools that normalize probability descriptions to minimize framing effects in LLM assistance for investment and risk management decisions.
Authors
Yuyao Zhang, Yakun Cui, Han Zhu, Jinsong Lu, Yike Guo, Sirui Han
Abstract
Large language models (LLMs) are increasingly used to support consequential decisions, yet their reliability under equivalent representations of uncertainty remains unclear. Existing work has documented prompt sensitivity, cognitive biases, and confidence miscalibration, but it is still unknown whether high-stakes advice changes when the same probability is expressed through complementary gain and loss frames. We introduce FrameBench-HPS, a controlled benchmark of 2,880 prompts derived from 240 high-stakes scenarios across healthcare, finance, education/career, and legal/consumer-rights domains. Across seven LLMs, gain-loss framing changes the recommendation category in 32.4% of complete groups, with most changes involving uncertainty rather than direct reversals. Human evaluation confirms corresponding shifts in perceived risk and directiveness. Robustness experiments show that framing effects persist under conversational paraphrases and exceed ordinary repeated-generation variability. Probability canonicalization substantially reduces framing sensitivity, identifying probability representation as a practical target for improving consistency in high-stakes LLM decision support.