DeltaTrace assigns clear support or opposition scores to language tokens
DeltaTrace: Efficient Signed Attribution for Reasoning Language Models
Machine Learning
Summary
When language models generate answers, it's important to know how each word in the input helped or hurt the result. The authors introduce DeltaTrace, a method that measures how removing each word changes the confidence in the model’s answer. It assigns positive or negative scores to each word, explaining their influence consistently and efficiently for different types of language models. This helps understand and trust model decisions better, especially in tasks that require reasoning or finding specific information.
What this means in practice
- •For natural language engineers: Provide clear, signed influence scores of input words to explain and debug language model outputs in reasoning tasks.
- •For ai platform operators: Improve trust and transparency in language model services by tracing word contributions to predicted answers for better auditability.
Authors
Chen Liang, Daoping Wu, Donghua Yang, Zhiyu Liang, Ying Zhang, Shuguang Xiong, Hongzhi Wang
Abstract
We study how words in the context support or oppose a language model's response and present DeltaTrace, a novel attribution framework that explains the effect of input tokens on a fixed model response. DeltaTrace measures how the response's log-probability changes when the context tokens being explained are masked. To distribute this difference among source tokens, it constructs operator-specific local rules for finite changes and composes them backward through the computational graph using effective secant slopes. Each token is assigned a signed attribution as the inner product of its embedding displacement and the corresponding backward-propagated slope. Local conservation makes these contributions sum to the measured score difference and preserves their meaning when aggregated over sentences or passages. DeltaTrace supports both attention and linear-attention models, with auxiliary memory growing linearly with sequence length at fixed model dimensions and chunk size. On the evaluated attention model, DeltaTrace achieves the lowest deletion-faithfulness scores (RISE and MAS) on most retrieval and reasoning tasks and the highest needle-in-a-haystack Recall, exceeding FlashTrace by 5.67-24.33 percentage points. On the evaluated hybrid attention and linear-attention model, DeltaTrace lowers RISE on 9 tasks and MAS on 12 relative to FlashTrace.