Papers for

debate platform operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Chinese language debate data offers new test for AI argument skills

Chinese Competitive Debating Dataset and Benchmark

Abstract: Debate adjudication requires tracking how arguments develop through interaction, yet existing datasets rarely combine fine-grained debate transcripts with professional judgments collected during real competitions under a shared rubric. We introduce a dataset and benchmark for evaluating large language models' understanding of competitive Chinese-language debate at the match, stage, and speaker levels. We organized 182 matches and recruited 120 professional judges, with each match independently adjudicated by three judges using a predefined rubric. After excluding matches with incomplete records, the dataset contains 148 matches, 2,698 stages, and 20,542 exchange units, with manually verified transcripts and segmentation. It preserves original stage scores, match votes, best-debater ballots, and adjudication rationales. We define three tasks: winner-tendency prediction, stage-score prediction, and best-debater prediction. Zero-shot evaluation of multiple large language models yields a highest winner-prediction accuracy of 66.2%, a highest Pearson correlation of 0.250 between model stage scores and mean human ratings, and a highest best-debater prediction accuracy of 56.8%. The dataset and benchmark provide a testbed for studying large language models' understanding of interactive argumentation and their agreement with professional judges.

Fri 18 SeptComputation and LanguageArtificial Intelligence
The gist
Judging debates is hard because it requires understanding how arguments change during the discussion. The authors collected detailed Chinese debate records judged by professionals in real competitions to create a new dataset. They used this data to test AI models on their ability to predict winners, scores, and top debaters. The best AI models were correct about two-thirds of the time at predicting winners but struggled more with the other tasks. This dataset helps study how well AI can understand complex, interactive arguments in Chinese debates.
Open → 2609.21637v1