Embedding Models for Stance-Aware Argument Retrieval
Computation and LanguageArtificial Intelligence
Summary
The authors study how computer models find arguments that support or oppose specific claims, which is important for making decisions based on those claims. They find that current models often focus too much on whether the argument talks about the same topic, ignoring if the argument actually agrees or disagrees with the claim. When the authors tried to fix this by training models differently, the models started paying too much attention to words like "supports" or "refutes" and missed the real meaning. To solve this, the authors introduced new ways to test this problem and used a balanced training method with examples that reverse the stance, helping models understand the logic better. Their approach improved how well models find arguments that both relate to the topic and have the correct stance.
Authors
Angelo Sparacino, Francesca Toni, Adam Dejl
Abstract
In computational argumentation, obtaining arguments that explicitly support or attack given claims is a critical precursor to downstream reasoning tasks. When these supporting and attacking arguments are to be retrieved using semantic search methods, they need to be assessed for topic-relevance to the claims of interest as well as for correctness of their (positive or negative) stance towards the claims. In this paper we explore how dense embedding models (hereafter, models), powering modern retrieval pipelines, can serve as the basis of semantic search incorporating this dual assessment. We show experimentally that existing models struggle with asymmetric reasoning, exhibiting a strong bias toward topical overlap while ignoring instructional stance. We also show that correcting this bias via contrastive training triggers a new failure mode where models over-correct, over-fixating on polarity keywords (e.g., "supports" or "refutes") at the expense of the semantic topic. We thus introduce diagnostic word-ablation metrics to quantify this phenomenon and propose a data-centric solution. By implementing a balanced argument curriculum alongside LLM-augmented, stance-inverted arguments, we force the (embedding) models to learn deeper directional logic rather than exploiting superficial lexical shortcuts. Our evaluation demonstrates that, for sufficiently powerful models, this approach can alleviate the observed overcorrection, achieving further improvements in stance-aware argument retrieval.