Anthropomorphic AI plans safe conversations to protect users
Sampling Safe Futures: Multimodal Trajectory Planning for Personalized Safety in Anthropomorphic AI
Human-Computer Interaction
Summary
Some AI systems act like social companions and remember personal details, which can sometimes lead to harm over time. This paper presents a way for AI to look ahead during chats and choose responses that keep the user safe in the long run, not just in a single reply. The authors use a method that predicts different possible conversation paths and picks the safest, most helpful one. Their tests show this approach reduces harmful interactions while keeping the chat useful.
What this means in practice
- •For chatbot developers: Design chatbots that anticipate conversation risks over many interactions to prevent user harm while maintaining helpfulness.
- •For customer support platform teams: Improve automated response strategies by filtering out unsafe paths in multi-turn conversations with customers.
Authors
Benedetta Picano, Dusit Niyato
Abstract
Anthropomorphic artificial intelligence systems increasingly remember personal details, display empathy, and are engaged with as social counterparts, creating forms of risk that emerge from the evolution of the user-system relationship over time. Existing safeguards largely operate at the level of individual conversational turns and cannot determine whether a sequence of seemingly acceptable interactions is cumulatively moving a particular user toward harm. This paper introduces personalized trajectory-level safety, a framework that treats relational safety as a sequential decision problem over a latent escalation state inferred from the user's messages and influenced by the system's responses. At each turn, a screening step first discards any response strategy that does not preserve at least one safe continuation of the interaction under every plausible model of the user. Among the remaining strategies, we formulate action selection as multimodal trajectory sampling, and use a Generative Flow Network to generate diverse future evolutions in proportion to their plausibility, safety, and utility. The system then selects the strategy that preserves the largest fraction of safe and useful continuations. We evaluate the framework in simulation, calibrated on statistics reported for real human-chatbot interactions, and using response strategies derived from public benchmarks. Results show that trajectory-aware decision making substantially reduces the frequency of harmful states while keeping helpful interaction. This work reframes safety for anthropomorphic AI from response-level filtering to personalized control over the future evolution of human-AI relationships. The source code is available at https://github.com/benedettapicano/ANTHROPOMORPHIC_SAFETY_TRAJ.