Existing prompt guards fail to stop attacks on legal contract AI tools
Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks
Artificial Intelligence
Summary
Artificial intelligence tools are sometimes tricked by sneaky instructions hidden in legal contracts. The authors tested eleven guard techniques meant to stop these tricks but found that many failed, with some incorrectly blocking up to 92% of real contracts. They discovered new ways these guards fail and showed that using smart language models to double-check can catch these attacks more reliably and cheaply than just using the best models alone.
What this means in practice
- •For legal tech developers: Improve document analysis AI by integrating cascaded model checks to reduce prompt injection errors in reviewing legal contracts.
- •For security teams in ai companies: Design more robust prompt injection guards based on detected failure modes and cascaded verification to better protect AI systems handling sensitive documents.
Authors
Mark Heimann, Katherine Taylor, Bruno Jacob
Abstract
Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal contracts and find that existing guards struggle: document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC; and a five-regex baseline is not Pareto-dominated by any guard in the panel. Controlled probes locate model-specific as well as general failure modes. Prompted LLMs resolve the tested failures, including a second threat category (unauthorized scope claims) on which every guard scores zero. A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.