AI-generated debates express values differently than humans do
Value-Rich, Differently Organized: Measuring Value Expression and Organization in AI-Generated Debates
Artificial Intelligence
Summary
Current ways to judge AI debates focus on how convincing they are but miss how values are shown and organized. This paper introduces a way to measure what values appear in debates and how they are spread across parts of the argument. Using a theory of basic human values, the authors compare human and AI debates, finding AI debates mention as many values but organize them less clearly across turns. This helps show that judging AI debates needs to look beyond just which values appear to how they are presented.
What this means in practice
- •For ai system developers: Improve AI debate systems by analyzing how values are expressed and organized to produce more coherent value-driven arguments.
- •For content moderation teams: Assess AI-generated debates for value clarity to better detect subtle biases and inconsistencies in automated arguments.
Authors
Maoyuan Li, Haoyuan Li, Zhongsheng Wang, Jiamou Liu
Abstract
Current evaluation of LLM-generated debate largely focuses on argumentative quality, leaving the value structure of generated arguments underexamined. We introduce a diagnostic metric framework for comparing how values are expressed and organized in human-reference and AI-generated debates. The framework separates value expression, the range of recognizable values a debate invokes, from value organization, how those values are distributed and sustained across turns. Grounded in Schwartz's Basic-10 value theory, it represents debates as multi-level value distributions over argumentative units, utterances, stances, and full debates, enabling metrics for value coverage, alignment, diffusion, and cross-turn dynamics. Across multiple generation systems, AI-generated debates are broadly value-expressive, often activating as many or more value dimensions than matched human references, but they organize these values differently. Their utterances are more locally value-diffuse, and source-blind judges distinguish value clarity from value focus. These findings show that debate evaluation should consider not only value presence, but also value organization. Code and supplementary materials are available at https://github.com/Maoyuan-li/Value-Expression-and-Organization.