Ai counsel evaluated on faithfulness to user religious beliefs
FaithfulBench: Does AI Counsel Uphold or Undermine the User's Professed Faith?
Human-Computer InteractionArtificial IntelligenceComputation and Language
Summary
This paper looks at whether AI assistants give advice that matches a person's religious beliefs. The authors created a test called FaithfulBench that checks if AI follows the specific faith tradition of its user when discussing moral questions. They tested five AI models under different conditions, like knowing or not knowing the user's religion, and whether the AI stuck to faith-based answers even under pressure. The results show that simply knowing the user's religion helps, but providing AI with a guide based on religious texts improves faithfulness and consistency.
What this means in practice
- •For chatbot developers: Develop AI advisors that respect and adhere to users' specific religious beliefs in moral or ethical discussions.
- •For digital therapy teams: Improve AI counseling systems to better maintain users' faith-based values when providing moral guidance.
Authors
M Waleed Kadous, Benjamin Olsen, Walter Scheirer, Daniel D. Slate, Alexander Arnold, DZ Kalman
Abstract
Do AI assistants help believers reason about moral dilemmas consistently with their faith? We present FaithfulBench, the first benchmark to score AI counsel across traditions by how well it adheres to the user's professed faith. Scenarios are drawn from each tradition's most respected texts, with the faithful answer known and applied by the judges as the standard. We test five frontier models under three conditions: the AI does not know the user's tradition; it receives a one-line prompt identifying the user as a practicing adherent; or it receives a companion-counselor guide rooted in the tradition's sources. Two judges score the initial response and whether the model caves or holds when pressured toward the answer the user wants. When the tradition is unstated, models counsel from a secular therapeutic default and every model fails some believers. Naming the faith wins a faithful first answer but not steadfastness; the guide improves both.