Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA
2026-06-02 • Computation and Language
Computation and LanguageInformation Retrieval
AI summaryⓘ
The authors studied how to improve systems that answer legal questions by finding and citing relevant text passages. They found that just using similarity to rank passages doesn't work well and can be worse than picking randomly. To fix this, they trained a new lightweight model to reorder passages based on how important they actually are to the final answer, using a method called perturbation attribution. This improved how well the system's citations matched expert references, and the new approach worked across different language models. Their work shows that using attribution scores can help make better and more reliable passage selection.
Retrieval-augmented generationLegal question answeringSemantic similarityPerturbation-based attributionC-LIMECross-encoderPassage re-rankingCitation faithfulnessAQuAECHR benchmarkLanguage models
Authors
Mohamed Hesham Elganayni, Selim Saleh
Abstract
Retrieval-augmented generation systems for legal question answering typically retrieve passages based on semantic similarity and provide them to a language model, which then generates cited answers. Prior work assumes that highly ranked passages are most likely to be usefully cited by the model. Perturbation-based attribution methods, such as C-LIME, have been used exclusively for post-hoc explanation. However, on the AQuAECHR benchmark, semantic similarity does not correlate with passage attribution. Within a retriever's candidate pool, similarity-based ranking performs worse than random selection at surfacing gold citation paragraphs. To address this limitation, a lightweight cross-encoder is trained on continuous perturbation-based attribution scores to re-rank passages prior to generation. This approach is evaluated on the AQuAECHR benchmark, using two language models and five-fold cross-validation. The re-ranker substantially improves citation faithfulness and alignment with gold expert answers. Notably, two re-rankers trained independently on different models converge beyond their raw attribution agreement. This finding indicates that the cross-encoder reduces model-specific noise and produces a shared relevance signal that partially transfers across models, although same-model re-ranking remains more effective. These results demonstrate that perturbation-based attribution provides a practical, model-agnostic training signal for citation-aware retrieval.