Attribute-Based Activation Steering of LLMs for Group-Specific Explanation Generation
Computation and Language
Summary
The authors studied how to make large language models (LLMs) explain things better to different groups of people by tailoring explanations to their knowledge and style. They created a method that adjusts the model’s internal workings to match group-specific traits, rather than just giving different prompts. Their experiments showed this approach makes explanations more targeted and still accurate. Experts from various groups confirmed the improved tailoring over existing methods.
Authors
Leandra Fichtel, Janek Prange, Henning Wachsmuth
Abstract
To effectively enable people to understand new topics, explanations should be tailored to their backgrounds and abilities. So far, prompting alone has been shown to be insufficient for creating such explanations and other computational methods are missing. Therefore, this paper investigates whether LLMs can be steered to generate explanations that are tailored to a specific group of people. To this end, we propose an approach that first identifies group-specific attributes in terms of explanatory style and knowledge of a specific target group. Building on activation engineering, it then computes attribute-based steering vectors and adds them to the internal activations of an LLM during inference to enable a fine-grained steering. In our experiments, we assess the steering effectiveness of our approach in terms of specificity and factuality of the generated explanations. Additionally, we evaluate the explanations in a study with human experts from different target groups. Compared to prompting and state-of-the-art steering baselines, our approach tailors the explanations significantly better to the target group while largely maintaining factuality.