Large language models perform better when personas match their knowledge
Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models
Computation and Language
Summary
Sometimes, asking language AI to pretend to be a certain character helps it answer questions better. But this doesn’t always work because it depends on how well the AI understands the role it is playing. The researchers found that the success of this trick changes depending on the AI’s size, the topic, and the language used. They came up with a way to mix role prompts in several languages to improve the AI’s understanding and make it perform better across the board.
What this means in practice
- •For ai system developers: Create more reliable language model prompts by mixing multiple language versions of role instructions to boost performance in diverse domains.
- •For multilingual chatbot builders: Improve chatbot understanding and responses by using multilingual concatenated prompts that better align with persona roles and knowledge.
Authors
Xingjie Zhuang, Jialong Tang, Chulun Zhou, Buchao Zhan, Zhirui Li, Junhui Li, Yazheng Yang, Jinsong Su
Abstract
Role-playing prompting has become a popular yet simple technique for improving LLM reasoning and output quality. However, whether it consistently boosts performance across diverse domains remains unclear, as systematic validation is lacking. To fill this gap, we run multi-model, cross-domain, and multilingual experiments on MMLU and MMLU-Redux. We find that gains from role-play prompting depend heavily on model capacity, knowledge domain, and prompt language. Drawing on metacognition theory, we propose the persona-related cognitive alignment hypothesis: role-play works only when the LLM correctly grasps the designated persona and its associated knowledge domain. We test this hypothesis through persona information richness ablation, layer-wise entropy divergence analysis, and latent thought-space deflection observation. To reduce persona cognitive bias and stabilize role-play performance, we propose \textbf{M}ixed-\textbf{L}anguage \textbf{C}oncatenate \textbf{P}rediction \textbf{(MLCP}), a simple, training-free, and efficient multilingual prompt concatenation strategy. It aggregates semantically equivalent role prompts to enrich complementary representational cues. Extensive experiments show that MLCP consistently outperforms vanilla role-play prompting across all tested LLMs.