Papers for

software maintaining ai systems

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Addressed memory updates improve targeted knowledge injection in language models

Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory

Abstract: Addressed conditional memory provides a local surface for post-hoc knowledge injection, but a successful write need not transfer to a new question or preserve untargeted behavior. This study combines a controlled GPT-2 comparison with production-memory experiments on Qwen3.8-Flash-Next-FP8 and DeepSeek-V4.1-Flash. Newly initialized modules attached to frozen GPT-2 are tested in a 2 x 2 design: row-only versus row-plus-projection updates, each with or without fixed module-residual anchoring. All arms reproduce 50 trained completions in each of three sampling seeds. Natural-question target-string hits average 5.3% for both row-only arms, 28.7% for unanchored projection updates, and 22.0% with anchoring, while strict answer scores remain zero. Anchoring lowers measured drift; lexical hits do not establish entity-specific retrieval. In the archived Qwen experiments, gradient writes make the designated first answer token top-ranked for 132 independently reset targets, comprising 93 new promotions and 39 already top-ranked targets. A separate matched 24-target comparison gives 13 new promotions with Adam versus two with SPSA at equal iteration counts but unequal computational procedures. An 80-row case includes saved continuations and a replay record, while reformulations bound its scope. Six DeepSeek zero-order cases yield one new target-token promotion and one regression. These results support bounded writing feasibility and separate trained expression, question-form behavior, and preservation; first-token rankings do not establish complete-answer success or universal conditions for writable memory.

Thu 15 OctComputation and Language
The gist
The paper studies how to add new knowledge to a language model’s memory after it is trained, so it answers new questions correctly without messing up old answers. The authors tested different methods of updating parts of the model and found some ways to improve recall for specific target answers, but the updates didn't guarantee full correct answers or perfect behavior. They showed it is possible to make controlled changes to memory in models like GPT-2 and larger Qwen and DeepSeek models, but the process is limited and complex. This work helps understand how injected knowledge behaves in model memory and what challenges remain.
Open → 2610.21553v1