Vietnamese legal questions benchmark challenges retrieval and answering
ViLegalExpert: A Large-Scale Benchmark for Vietnamese Legal Retrieval and Question Answering from Real-World Consultations
Computation and Language
Summary
It is hard for AI to answer legal questions in a way that clearly shows the trusted laws and rules they come from. The authors created a big set of real questions and answers from Vietnamese legal consultations, covering many areas of law. This set helps test how well AI can find legal documents and provide correct answers based on them. They found that even the best AI struggles to find the right evidence and give grounded answers. This work gives a tough test for building reliable legal AI in Vietnamese.
What this means in practice
- •For legal technology developers: Build and evaluate Vietnamese legal AI systems that answer citizen questions with verified lawyer evidence across many legal domains.
- •For customer service teams: Improve automated support for Vietnamese legal inquiry handling by grounding responses in real-world lawyer-verified answers.
Authors
Dat Tien Nguyen, Nghia Hieu Nguyen, Anh Thi-Hoang Nguyen, Dung Ha Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
Abstract
Trustworthy Legal AI requires systems that can answer legal questions while grounding their responses in authoritative sources. However, existing Vietnamese legal benchmarks provide limited coverage of real-world legal consultations. We introduce \textbf{ViLegalExpert}, a large-scale benchmark constructed from authentic citizen--lawyer consultations, containing over \textbf{172K} questions across \textbf{34 legal domains}, together with professional answers and expert-verified legal evidence. ViLegalExpert supports legal information retrieval, extractive QA, and abstractive QA. Experiments with representative retrieval methods and language models reveal substantial challenges in evidence retrieval and grounded answer generation. While pretrained models perform strongly on QA, hybrid retrieval achieves the best retrieval performance. These results demonstrate the difficulty of mapping naturally expressed legal questions to authoritative provisions and establish ViLegalExpert as a challenging benchmark for reliable Vietnamese Legal AI.