Papers for

investment advisory firms

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Ai financial advice shifts with changing self reported risk levels

Same Person, Different Advice: Auditing LLM Financial Advice with Repeated Risk-Tolerance Responses

Abstract: AI financial advisers set a client's stock allocation largely from one stated-risk question, yet people do not answer it the same way twice: in a national US panel, 41% of households chose a different answer two years later, and in a laboratory retest a third of people moved two or more points on a 0-10 scale after six months. Seven large language models from five providers gave one-turn allocation advice for 800 real household profiles in which only the risk answer changed. The recommended stock share moved by 10 to 16 percentage points on average, and by 24 to 35 when the answer changed, against 0.6 to 4.2 points for the same prompt asked twice; in the laboratory retest the rating alone moved advice by 11 to 18 points. Without any note, the models respond to the answer about as strongly as when told that it is very reliable (less so for the two Claude models). If that instructed response is how each model maps attitude to advice, discounting the answer for its measured reliability, as classical test theory prescribes, would cut the swings by a factor of 2.4 to 3.3. The swing is largest for clients under 40. A note that the answer is noisy steadies some models, but shifts their weight to the client's age and pulls cautious clients towards a generic middle allocation: clients unwilling to take any risk are advised 22% stocks instead of 4%. Three extra money questions reduce the swing for some models, with no statistically detectable change relative to how much advice differs across people. In a scripted test, one clarifying question removes nearly all of the swing from an answer the client then corrects (33.5 points to 1.9 on average) while keeping the response to a change the client confirms. We argue that same-person retest audits should be a standard check for AI advisers, and that what to do with an unreliable answer is a question of client autonomy as well as of statistics.

Fri 16 OctComputation and LanguageComputers and Society
The gist
People often give different answers about how much financial risk they want to take when asked the same question at different times. The authors tested multiple large language models that give stock advice based on one risk answer and saw that the recommended stock amounts changed a lot when the risk answer changed. They found that models treat risk answers as very reliable even though people’s self-reports can be inconsistent. Adding extra questions or clarifying responses helps stabilize the advice. The authors suggest checking AI advisers by repeating risk questions to better handle unreliable answers.
Open → 2610.23060v1