Reflection agent demands weaken career plan commitment and increase doubt
Faithful Where It Can Be Checked: Auditing a Reflection Agent Against Its System Prompt in a Randomized Trial
Human-Computer InteractionComputers and Society
Summary
Some computerized helpers are designed to guide people in thinking about their careers. The authors studied one using a large set of conversations and surveys. They found the helper often broke simple rules like not flattering and not pushing too hard. When it kept asking people to decide despite hesitation, those people ended up more doubtful about their career plans. These results suggest how to build better helpers that follow rules visibly and avoid pressuring users.
What this means in practice
- •For chatbot developers: Design reflection agents that visibly follow prompts and limit repeated decision requests to reduce user doubt.
- •For career counseling platforms: Create interactive career reflection tools that avoid pressuring users repeatedly to decide, based on conversational audit findings.
Authors
Subigya K. Nepal, Serena Soh, Noah Vinoya, SoHyun Park, Mahnaz Roshanaei, Gabriella Harari
Abstract
Conversational agents are increasingly used to guide reflection. A recent randomized trial compared a GPT-4o career reflection agent with the same program in a static journaling survey. Agent participants ended less committed to their career plans and more doubtful. We coded all 17,930 turns from its two studies, checked our coding against human coders and linked conversations to the trial's surveys. The rules the agent followed were the easy-to-check ones, like a reply length cap. Told not to flatter, it praised participants in half of its turns; told to challenge gently, it almost never did, and such a break leaves no visible trace. The behavior tied to the worse outcome was the demand to decide: the survey posed each decision once, while the agent asked again when participants hesitated, and those pressed most ended most doubtful. Our findings inform reflection agent design and the writing of checkable instructions.