System One model adapters can change final decisions unexpectedly
DICE: Tilting the Odds through Interface Attackson System One Model
Cryptography and SecuritySoftware Engineering
Summary
System One models give answers based on the information they receive, but how other parts of an application handle those answers can change the final decision. The authors studied how three different points—taking data, asking questions, and interpreting answers—can be altered to shift decisions, even though the core model and its real outputs can’t be controlled by an attacker. They found different ways the connection between what the model means and what the application does can break down. Some checks helped stop these changes but also sometimes blocked good decisions.
What this means in practice
- •For application developers: Improve security checks in interfaces that handle model outputs to prevent unintended decision shifts caused by data extraction or interpretation layers.
- •For ai system integrators: Design better adapter components in AI pipelines to ensure final decisions remain consistent with original model judgments despite transformations.
Tested on simulated data.
Authors
Ben Dong, Tianyi Lu, Zhonghao Guo, Qian Wang
Abstract
System One models offer typed judgments that applications can turn into decisions. Their answers, however, are conditional on the state and question supplied by surrounding components, and on how the application uses the response. We study this boundary in a Jev-based pipeline with bounded substitutions at three adapter outputs: state extraction, question generation, and output interpretation. Across three small synthetic studies, each adapter can change the final decision while Jev and its authentic API responses remain outside attacker control. The cases expose distinct failures of source fidelity, answer meaning, and response relevance. Checks that bind these relationships stopped the corresponding interventions we tested, although source verification also rejected some valid clean decisions. The pilot identifies where correct request-level judgments can lose their connection to the original task; it does not estimate the prevalence of these failures in deployed systems.