TrustJev improves policy decision safety against instruction attacks
TrustJev: Separating Policy Authority from Untrusted Evidence in LLM-Based Decisions
Artificial Intelligence
Summary
Sometimes software needs to decide what to do based on facts given in natural language. But the facts it receives might also try to trick it into ignoring its rules. The authors built TrustJev, a system that carefully separates trusted rules from potentially tricky evidence to keep decisions correct. Their tests show TrustJev better resists attacks that try to override rules without losing accuracy on normal inputs.
What this means in practice
- •For software security teams: Protect policy-based decision systems from malicious instructions hidden in input data using TrustJev’s bounded correction approach.
- •For access control system developers: Improve reliability in authorization decisions by separating trusted policy evaluation from untrusted evidence to resist override attacks.
Authors
Zhiwei Xu, Xuyang Liu, Xiaolin Peng, Weixian Deng, Jiabao Gao, Zirui Li, Tian Qiu, Boya Zhou, Wei Jiao, Hai Wan, Xibin Zhao
Abstract
Decision models enable software to select from predefined actions using natural language inputs. However, the external evidence that supplies facts for these decisions may also contain instructions that attempt to override the application's policy. We propose TrustJev, a framework combining protected policy scoring with bounded evidence corrections. A single Transformer backbone computes policy scores and a correction budget from trusted policy and candidates, then bounds evidence corrections by that budget. Paired supervision encourages stable predictions under instruction insertions that preserve task facts and changed preferences when edits change the correct answer. Authority supervision accommodates evidence when the policy alone cannot determine the decision. We evaluate a 0.6B implementation against 14 baselines on three datasets covering authentication, instruction hierarchy, and executable authorization rules. TrustJev achieves the highest mean Balance, the harmonic mean of accuracy on non-attack and attack inputs, exceeding the strongest baseline on each by 0.57-1.05 percentage points. In the authorization benchmark, it correctly answers all 32 attacks per source in a fixed pool for 95.77% of sources, compared with 80.98% for the strongest comparator. TrustJev maintains comparable inference memory and median latency, with additional overhead primarily during training.