Psychological words cause problems in AI agent governance
The Disciplinary Language Transfer Problem: How Psychological Vocabulary Produces Governance Failures in AI Agent Deployment
Computers and SocietyArtificial Intelligence
Summary
Words like learning, memory, and trust are often used to describe AI, but these come from psychology and don't fit how current AI actually works. The authors explain that using this language creates confusion and failure when organizations try to manage or control AI systems. They identify deeper assumptions behind this vocabulary that don't apply to AI and propose a practical checklist and new vocabulary to better understand and govern AI agents. Fixing this language problem is essential for clear rules about AI.
What this means in practice
- •For ai governance teams: Use the six-question Disciplinary Audit to identify and correct misleading psychological terms in AI management policies for clearer oversight.
- •For organizational compliance officers: Guide policy updates by replacing psychological metaphors with accurate AI terms, improving accountability of automated agents in organizations.
A position paper. It proposes an approach and reports no results.
Authors
Kymberly Lasser-Chere, Tyler Akidau, Marc Millstone
Abstract
The vocabulary used to describe AI agents in governance contexts -- learning, memory, values, compliance, identity, trust -- is borrowed from psychological and organizational science, contributing to systematic failures in how organizations deploy, oversee, and hold agents accountable. This paper argues that the problem is not merely terminological but epistemological: psychological vocabulary carries an "invisible grammar" of its home discipline into governance discourse, calibrating frameworks to a metaphysical entity that does not exist in current AI architectures. We call this the disciplinary language transfer problem. Drawing on Wittgenstein's concept of language games, Kuhn's paradigm-laden observation, Haraway's situated knowledge, and Star and Griesemer's boundary object theory, we show that the transfer operates at three levels (epistemological assumptions, theoretical constructs, and surface vocabulary), each requiring a different remediation. We characterize six foundational epistemological assumptions embedded in Western psychological governance discourse, trace their origin in specific philosophical traditions, and show why each fails when applied to systems without developmental continuity. The paper's practical output is an actionable Disciplinary Audit: a six-question governance document scan operationalized through a translation taxonomy of thirty-seven terms mapping operational constructs to agent-appropriate replacements, presented here in abridged form and openly archived in full. The vocabulary reform proposed here is not merely terminological; it is the condition of possibility for governance frameworks that correctly identify what they are governing.