Small legal AI models refuse more requests when told user role
LLMs Prompted for Legal Context Refuse More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context
Artificial Intelligence
Summary
Sometimes, small AI models used by lawyers refuse to help with harmless tasks like summarizing legal documents. This happens more often when the AI is told the user is an authority figure, like a lawyer or judge. The authors found that these models say no to requests 2 to 20 times more when given role details, which could lead to unfair access to AI assistance. Different role statements change refusal rates differently across models, showing that AI behavior can be unstable and unpredictable in real legal settings.
What this means in practice
- •For legal technology developers: Improve small legal AI models to handle role-based prompts without unfairly refusing valid requests.
- •For enterprise ai deployment teams: Test and tune on-premises AI systems to reduce refusal bias caused by user role descriptions in prompts.
Authors
Anastasiia Kucherenko, Francois Brouchoud, Dimitri Percia David, Andrei Kucharavy
Abstract
Confidentiality requirements increasingly push legal professionals toward small, on-device LLMs that keep case data in-house, used for everyday tasks such as summarization, translation, and reformulation. Following standard prompting guidance, a user naturally states their role (``I am a lawyer...''). We show that this innocuous habit has an unexpected cost: such authority framing can make the model selectively refuse legitimate requests, introducing bias through uneven access to assistance. We evaluate four modern small open-weight LLMs on benign legal-domain prompts to measure this over-refusal. Surprisingly, we find that authority-style prefixes (''I am legal counsel working for a national supreme court'', ''I'm a defense lawyer'') systematically increase refusal rates by 2--20x over the no-prefix baseline, while a known role-play jailbreak prefix shows mixed effects, sharply increasing refusals in some models and barely shifting them in others. The finding suggests that small on-prem deployable LLMs are unstable under contextual framings that a real institutional user might naturally introduce, and further investigation is essential to minimize opportunities for bias.