Trustworthy delegation needs clear reasons for ai decisions
Cybernetic and Epistemic: A Missing Vocabulary for Trustworthy Agentic Delegation
Artificial IntelligenceHuman-Computer Interaction
Summary
When people rely on AI systems to write code or make decisions, simply checking if the AI's output is approved isn't enough. The authors explain that language used between humans and AI has two roles: helping coordinate actions and helping share understanding. They argue that good oversight requires clear reasons behind AI decisions that others can check, instead of just rubber-stamping approvals. They propose a way to record decisions so that someone else can test what would happen if conditions changed, making AI behavior more accountable.
What this means in practice
- •For software development teams: Create delegation records for AI code generation that include conditions under which decisions would differ, enabling accountable supervision.
- •For legal compliance officers: Audit AI decision-making processes using reconstructed conditions to verify responsible governance and prevent rubber-stamping in regulated environments.
A position paper. It proposes an approach and reports no results.
Authors
Jérémie Lumbroso
Abstract
As code generation is increasingly delegated to AI systems, the bottleneck is shifting from writing code to supervising the systems that write it --- a shift CS-education researchers have begun to name. This shift exposes a vocabulary gap: the field asks for "human oversight" without a working distinction between the two things language does in a delegation channel --- coordinate action (cybernetic: words succeed when the world comes to match them) and coordinate understanding (epistemic: they succeed when they answer to the world and a hearer can check that they do). The failure this names is not cybernetic language but epistemic-form language doing cybernetic work: explanation-shaped output calibrated for approval rather than truth. Oversight that checks only whether an output was approved is satisfiable by rubber-stamping; oversight that holds an agent accountable requires the reasoning behind its work be retrievable and checkable. We present three delegation episodes --- illustrations, not controlled evidence --- in which epistemic engagement proved practicable while remaining auditable, one public record where a recommendation was withdrawn on its own stated terms, and one failure case illustrating oversight that requires no reasons for its discretionary choices. We propose a criterion for agentic-system governance, alongside existing technical trust properties: every consequential choice should carry the condition under which it would have gone otherwise, in a form a third party can test. Without such a condition, a third party cannot distinguish a decision from a rubber stamp. We give the criterion an operational form --- a two-part reconstruction test scoring a delegation record by whether a second reader can predict what the agent does under a perturbation --- and a deliberation-recording convention, ORRCF, that makes the condition a required component of every recorded choice.