Summary
Conversational assistants often struggle to remember changing user requests during a chat, sometimes forgetting rules or wrongly applying canceled ones. The authors propose StateWise, a method that teaches a language model to keep an updated list of rules it must follow, including tracking which ones are removed. They fine-tune the model to summarize and respond based on these changing rules, and further improve obedience by teaching it to prefer responses that follow updated instructions. This approach significantly reduces the mistakes where the assistant ignores changes in user requests, leading to better conversation results without extra tools at run time.
What this means in practice
- •For chatbot developers: Create conversational assistants that better remember and follow evolving user instructions across multiple turns.
- •For customer support platforms: Improve automated agents' ability to track and adjust to user requests within ongoing support conversations.
Abstract
Users of conversational assistants rarely state all requirements up front: they add constraints, amend them, withdraw them, digress, and return. Instruction-tuned LLMs fail here in two ways: active constraints stop being honored as turns accumulate (persistence failure), and withdrawn constraints keep being enacted (relapse). The only existing repair we know of is an external ledger that re-injects a compiled specification every turn. We propose StateWise, a self-improving approach that teaches a single LLM to maintain its own revisable constraint state, a typed state with tombstones for revoked constraints, so persistence and relapse are measured by executable checkers and ground-truth states are compiled for free. StateWise fine-tunes the LLM on two complementary skills, summarizing a dialogue into its constraint state and responding conditioned on that state, then self-trains on its own trajectories, keeping only turns that pass an exact round-trip check against the compiled state. This teaches 7-8B models to track the state almost perfectly but, surprisingly, not to obey it: on Llama-3.1-8B every supervised variant roughly doubles relapse. We therefore add a preference stage on self-generated relapse contrasts, where the rejected response enacts a revoked rule and the chosen one passes the round-trip check. On two backbones with held-out constraint families and unseen phrasings, StateWise cuts relapse by 8-22 points relative to supervised self-training, lifts conversation success by 8-19 points over the backbone, and beats an oracle compiled-ledger baseline on relapse with no inference-time scaffolding. We also characterize the recipe's cost on general instruction-following benchmarks and study weight interpolation as a mitigation.