Optimizing document endings to guide AI answers on facts
Shaping the Document, Not the Model: Optimized Suffixes for Context Following
Computation and Language
Summary
Sometimes AI language models ignore the information given in a document and rely on their own internal knowledge, even if it's wrong. The authors found a way to add special endings to documents that help the AI follow the document's facts instead. They do this by searching for the best sequence of words to add after the document, without changing the AI itself or giving extra instructions. This technique helped AI models better answer questions based on the document, even when the document contradicts what the AI originally 'knew.'
What this means in practice
- •For ai product developers: Add optimized document suffixes to improve AI answer accuracy without changing model or instructions.$Commercial implications: Enables selling better AI-based document Q&A or chatbot products that leverage existing models but deliver more reliable context following.
- •For content management teams: Enhance contextual reliability by appending optimized suffixes to user documents before feeding them to AI models.
Authors
Gia-Bao Dinh Ho, Sekh Mainul Islam, Masahiro Kaneko, Timothy Baldwin, Isabelle Augenstein
Abstract
When a user supplies a document and wants the answer drawn from it, a language model is expected to follow it even when it contradicts its parametric knowledge. Prior work has focused on making the model rely on context by steering activations, contrasting logits during decoding, or instructing it in plain language. We ask whether the document alone, with no instruction written into it and no access to the model at inference time, is enough. We optimize a document suffix by discrete token search, starting from the answer to a single question and extending the objective to a set of passages, to multi-hop questions that follow from the planted fact, and to later turns of a conversation. On counterfactual passages that contradict parametric knowledge of the model, the suffix flips nearly all items the model resists across three instruction-tuned model families, in most settings outperforms the decoding baseline and the strongest instruction we could write, and matches activation steering. Controls show that the suffix largely acts through the passage rather than injecting the answer, and a single suffix per model transfers to unseen passages. Each extended objective carries the effect to its target, so the objective, not only the document, sets how far the effect reaches.