Think Inside the Chunk: RegulaRAG for Regulation-Compliant Scenario Generation using LLMs: A Case Study of UN Regulation No. 152
2026-08-17 • Artificial Intelligence
Artificial IntelligenceInformation Retrieval
AI summaryⓘ
The authors developed RegulaRAG, a method to help generate test scenarios that follow complex automotive safety rules by better organizing and searching through large regulatory documents. They designed a smart way to split and enrich the text and then find the most relevant parts for creating these scenarios. When tested on a real dataset of vehicle safety regulations, RegulaRAG performed much better than other similar systems, even when given a lot of distracting information. Their approach stays accurate and efficient with large amounts of regulatory text, unlike other methods that struggle as the documents get bigger.
Retrieval-Augmented GenerationLarge Language ModelsAutomotive Safety RegulationsUN Regulation No. 152Test Scenario GenerationGraph TraversalInformation RetrievalDocument ChunkingRobustness TestingPenalized Scoring Metric
Authors
Vahid Zolfaghari, Nenad Petrovic, AndrÉ Schamschurko, Alois Knoll
Abstract
Generating regulation-compliant test scenarios is essential for validating safety-critical automotive systems, yet Large Language Models (LLMs) struggle to ground outputs in long, hierarchical standards. We present RegulaRAG, a Retrieval-Augmented Generation (RAG) pipeline that couples SmartChunking, reference-aware enrichment of paragraphs and tables via graph traversal, with Smart Retrieve & Rerank over these enriched units. To test our system, we evaluate on a manually curated dataset covering all scenarios in UN Regulation No. 152 (AEBS). Our study comprises: (i) a three-step progressive search that identifies near-optimal retrieval parameters without exhaustive grid search; (ii) head-to-head comparisons against five baseline RAG systems; and (iii) a robustness stress test that scales the source corpus with distractor content. Outputs are evaluated using a customized penalized scoring metric. Across all experiments, RegulaRAG achieves the highest average Meta-Score (82.99), outperforming the next-best system by 43% (NoRAG: 57.94), while operating at 14k-25k tokens per query versus up to 500k for graphcentric baselines. It maintains strong performance, remaining stable even as the number of regulatory sources grows, whereas competing RAG systems degrade sharply in both quality and robustness.