Framework activates targeted correction adapters for safer output
An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS
Machine LearningArtificial IntelligenceComputation and LanguageComputers and Society
Summary
Large Language Models sometimes say things that are biased or harmful. The authors created a system that adds small correction modules to the model which turn on only when needed to fix specific issues like toxicity or bias. This system chooses which correction to use as the model writes, making responses safer without slowing down or hurting how well the model does its tasks. Their approach is flexible and efficient, fitting onto existing models without retraining the whole thing.
What this means in practice
- •For ai product developers: Integrate dynamic harm mitigation adapters into chatbots to reduce biased or toxic responses without extensive retraining.$Commercial implications: Allows AI product teams to build safer, adaptive conversational agents with efficient modular fixes enhancing user trust.
- •For content moderation teams: Deploy targeted mitigation modules to automatically filter harmful content generated by language models during live interactions.
Authors
Roberto Campbell, Momin Abbass, Muneeza Azmat, Michal Ulewicz, Raya Horesh, Kristjan Greenewald, Rogério Abreu de Paula, Nathalie Baracaldo
Abstract
Large Language Models (LLMs) are powerful zero-shot learners but remain prone to misalignment with human preferences, often producing biased, toxic, or otherwise harmful outputs. Existing alignment methods, while effective, are costly and tightly coupled to the model, limiting flexibility and scalability. We propose a modular correction framework that augments pretrained LLMs with Activated LoRA (aLoRA) adapters and a context-aware routing mechanism to eliminate harms from misaligned model responses. Our approach enables expert adapters to activate mid-sequence without invalidating the KV cache, allowing low-latency, targeted correction during generation. Each expert is trained to detect and mitigate specific harms, such as bias or toxicity. A learned router dynamically selects appropriate experts based on the models intermediate outputs. We demonstrate that our system improves alignment on standard safety benchmarks while preserving task performance, offering a lightweight and efficient path toward safer and more controllable LLM deployments.