Do User-Authored Permission Policies Improve Protection Against AI Agent Overreach?
2026-08-27 • Human-Computer Interaction
Human-Computer InteractionCryptography and Security
AI summaryⓘ
The authors studied how people without software backgrounds control AI agents that act on their digital data by setting rules in advance versus deciding each time an action happens. They found that pre-set rules reduced the number of prompts but did not significantly save time overall, and these rules allowed more risky actions to go through compared to reviewing actions individually. Many users chose a cautious "ask" option in their rules, which meant they still made most decisions in the moment rather than fully trusting the preset rules. This shows that while preset rules can help reduce interruptions, people still prefer making decisions case-by-case for better control.
AI agentslanguage modeluser-authored ruleshuman-in-the-loopoverreach actionsconsequence categoriesruntime promptsautomation policydigital productsuser control
Authors
Ting Yan
Abstract
AI agents are poised to become a primary interface to digital products, acting across email, files, payments, and personal data. People without professional software backgrounds need understandable, reusable ways to control actions across services. We examine a mechanism in which a language model maps actions to plain-language consequence categories with user-authored "allow", "ask", or "never" rules. We ask what is gained and lost when decisions are made in advance as reusable rules rather than separately for each action. We analyzed 113 participants without professional software backgrounds across three conditions: per-action human-in-the-loop approval (HITL), automated per-action model review (AUTO), or user-authored consequence policy (POLICY). Participants judged 2 examples in each of 4 consequence categories; POLICY participants then set one rule per category. All supervised an 18-action simulated day, including 7 overreach actions. POLICY blocked less overreach than HITL (-20.1 percentage points, 95% CI [-32.1, -8.1]) and AUTO (-14.5 points, 95% CI [-25.8, -3.2]). POLICY lowered runtime prompts from 18.0 to 10.9, but total intervention time was not reliably lower when rule setup was included. Exploratory analysis showed that participants chose "ask" for 114 of 140 POLICY rules, returning most overreach actions to runtime. Of the 148 overreach actions executed in POLICY, 133 followed human approval and 15 ran automatically under "allow" rules. Across all 7 overreach actions, POLICY had the highest approval rate. Counterintuitively, user-authored rules did not by themselves provide stronger protection: many actions outside users' original requests went through after users approved them. These results reveal a gap between preference and commitment: repeatedly choosing "ask" preserves case-by-case choice but prevents a standing policy from settling decisions in advance.