Combining natural language and modular prompts for adaptable ai alignment

Ready2Blend: From Natural-Language Instructions to Composable Alignment Prompts

Computation and LanguageArtificial Intelligence

Summary

Getting large language models (LLMs) to follow new instructions without forgetting old ones is hard. The authors created Ready2Blend, which uses special prompts linked to text instructions, letting models adapt by mixing these prompts without changing their core. This method works almost as well as retraining the model fully but is faster and more flexible, supporting personalized and combined instructions without extra training. It keeps the main model unchanged, speeding up updates while keeping past skills.

What this means in practice

  • For ai development teams: Enable AI systems to quickly incorporate new behavior requirements without retraining the entire model, using modular prompts for flexible updates.
  • For customer support platform engineers: Create chatbots that can blend multiple customer service policies expressed as natural language prompts without model retraining or forgetting previous instructions.$Commercial implications: This enables development of customizable AI chatbots that adapt faster and maintain consistent service, a commercially valuable feature for customer support products.

Authors

Jeesu Jung, Hwan Chang, Juseon Do, Jeonghwan Choi, Jinho Choo, Sungwoo Nam, S. K. Hong, Hwanjun Song

Abstract

Continual alignment requires LLMs to adapt to new requirements without forgetting previously acquired behaviors. Natural-language instructions are flexible and composable but offer only indirect control, whereas post-training provides stronger adaptation at the cost of repeated parameter updates. We introduce Ready2Blend, which combines the flexibility of natural language with learned alignment. AlignFormer maps each requirement to a fixed-length alignment prompt stored in a modular prompt bank, while the backbone and prior prompts remain frozen. Composability regularization transfers the semantic geometry of textual requirements into prompt space, enabling inference-time blending and reweighting. Across two practical continual alignment settings, Ready2Blend is the only frozen-backbone method that matches post-training-based alignment methods, reaching $93.1$-$98.5\%$ of a joint-training reference with competitive retention, while requiring only a few prompt tokens and up to $4.3\times$ less training time. Its modular design further enables weighted personalization and order-free composition without retraining. Code will be released upon acceptance.