Physical controls improve human oversight of automated systems
Bring Buttons Back: Physical Interfaces for the Age of Automation
Human-Computer Interaction
Summary
As everyday technology becomes more automated and controlled by AI, people find it harder to understand and influence what these systems do because controls are hidden behind screens or removed entirely. The authors propose bringing back physical controls like buttons and switches, but with a new twist: these controls move and respond on their own to show what the system is doing. This approach helps people better see, predict, and change automated actions by making the controls act as a two-way communication tool between humans and machines.
What this means in practice
- •For industrial control engineers: Use actuated physical controls to increase operator awareness and intervention ability in automated manufacturing systems.
- •For smart home device designers: Incorporate self-updating buttons and knobs to make household automation clearer and easier to override.$Commercial implications: Enables selling smarter physical control interfaces that enhance user trust and control in automated home products.
A position paper. It proposes an approach and reports no results.
Authors
David Goedicke, Donald Degraen, Tom Igoe, David Sirkin, Wendy Ju, Stefan Schneegass
Abstract
As AI starts to permeate everyday life, deployment focuses on embedding intelligence in the background, abstracting away controls, and leaving automated decision-making opaque, with few obvious opportunities for human intervention. Where interfaces remain, design has drifted toward abstract, screen-based control, replacing material interaction with menu navigation. These shifts weaken the coupling between human action and system state, eroding operators' ability to understand, anticipate, and intervene in automated systems. We introduce Physically Stateful Interfaces (PSIs), a design concept that re-imagines traditional controls, such as buttons, switches, and knobs, as actuated, bidirectional interface elements. Three foundational behaviors, Self/Reset, Resist/Hide, and Assert/Unhide, unify affordance, feedforward, and feedback into a single interaction point. Rather than static inputs whose impact and consequences must be read elsewhere on a screen, PSIs position physical controls as shared mediators between automated systems and human operators. As automated systems become increasingly capable, their interfaces need to be deliberately designed to center human agency, supporting deliberate choice while keeping automation legible, contestable, and overrideable.