Generative AI and manual review perform similarly for eDiscovery tasks
Comparing GenAI to Machine Learning and Manual Review for eDiscovery
Machine LearningArtificial IntelligenceInformation Retrieval
Summary
Finding important information in large collections of documents for legal cases is challenging. Keeling et al. compared a generative AI tool with machine learning and manual human review. They found that generative AI and manual review have similar effectiveness when you consider their different goals and trade-offs. Their analysis also shows that advanced machine learning methods can be even more efficient and accurate when given a small set of initial human decisions.
What this means in practice
- •For legal technology teams: Choose between generative AI and manual review workflows based on similar accuracy but different cost and precision trade-offs in eDiscovery projects.
- •For data science teams in legal firms: Implement machine-learning models with Continuous Active Learning using small labeled samples to improve eDiscovery effectiveness and efficiency.
Authors
Gordon V. Cormack, Maura R. Grossman, Jeremy Pickens
Abstract
Keeling et al. [Keeling2026Generative] report on a case study recently conducted by Redgrave LLP in which they assert that Relativity's Generative AI Review tool (aiR) achieved 88% recall while Relativity Assisted Learning (RAL), its machine-learning tool, achieved 64% recall. These results are not comparable because the methods employed have different objectives, because of inter-assessor disagreement, and because they represent different recall-precision and cost-benefit trade-offs. Our analysis of the Redgrave data indicates that, when these differences are taken into account, aiR and manual review are comparable. Furthermore, established machine-learning models employing Continuous Active Learning (CAL) show state-of-the-art effectiveness, and superior efficiency, when trained only on a sample of 2,000 first-pass-reviewer responsiveness determinations.