Compression effects impact arabic legal language model accuracy with and without context
Compression Through a Context Lens: Evaluating Adaptation, Quantization, and Distillation in Arabic Legal LLMs
Computation and Language
Summary
Large language models can be too big for some legal offices to run easily. The researchers studied ways to make these Arabic legal models smaller by adapting them, shrinking their memory use, and teaching smaller models to copy bigger ones. They found that these changes affect model accuracy differently depending on whether the model gets extra legal info (context) while answering. Testing with and without context helps understand how well these smaller models work in legal settings.
What this means in practice
- •For legal tech developers: Build smaller Arabic legal language models that maintain accuracy when legal context is unavailable.
- •For cloud infrastructure teams: Reduce memory requirements of Arabic legal AI models by applying four-bit quantization with minimal accuracy loss.
Authors
Md. Rafiul Biswas, George Mikros, Wajdi Zaghouani
Abstract
Model compression can facilitate the deployment of large language models in resource-constrained legal settings, but its effects on domain-specific performance may depend on the evaluation protocol. We investigate this issue using ArabLegalEval, a Saudi Arabic legal benchmark, under both with-context and no-context evaluation. The former provides supporting legal passages, while the latter assesses performance without external context. Removing the context produces accuracy gaps of up to 27 percentage points for the base model. We examine three stages: domain adaptation, quantization, and knowledge distillation across FANAR-9B, JAIS-8B, and Qwen2.5-7B. Four-bit quantization reduces model memory by 64-71%, with changes in no-context MCQ accuracy ranging from -3.2 to +1.6 percentage points. Distillation produces more variable results, with accuracy generally declining as student size decreases. A Qwen2.5-7B student distilled from FANAR achieves 76.6% no-context accuracy, compared with 77.2% for its teacher. Our findings suggest that no-context evaluation provides a useful complement to context-supported testing when assessing the effects of adaptation and compression on Arabic legal language models.