Summary
Making sure AI systems follow rules can be hard because companies describe their processes in words, but it's tough to turn that into clear lists of what is missing or needs fixing. The authors created a list of common technical compliance gaps and built a two-part AI framework that first finds specific issues from company answers, then suggests fixes ranked by how well they fit regulations and effort needed. They tested their system on many different AI systems and it found compliance gaps and fixes more accurately than previous automatic methods. This approach could help compliance experts by reducing their workload while keeping audits clear.
What this means in practice
- •For compliance officers: Automatically identify and rank missing controls in AI systems from questionnaire responses to streamline regulatory audits.
- •For legal technology teams: Integrate taxonomy-based AI gap analysis into tools that support compliance with evolving AI regulations and recommend prioritized fixes.$Commercial implications: Enables AI compliance software vendors to offer automated gap detection and remediation planning as a premium feature.
Authors
Ilias Driouich, Hongliu Cao, Eoin Thomas, Yannick Devaux, Agnes Brodelle
Abstract
Assessing regulatory compliance for high-risk AI systems requires systematic identification of technical gaps such as missing controls, in- adequate processes, or shallow evaluation of practices. Compliance officers typically admin- ister structured questionnaires collecting evi- dence of current practices, where organizations respond with descriptions of their existing pro- cesses, technical capabilities, and architectural choices. However, translating these compli- ance questionnaire responses into typed, ac- tionable gap diagnoses remains unsystematic. In this paper, we present a literature-grounded taxonomy of technical compliance gaps cover- ing areas such as accuracy, robustness, logging, and human oversight, and embed it in GapTo- Compliance, a two-stage agentic framework coordinating two specialized agents: a Gap Extraction Agent that performs hierarchical rea- soning from compliance areas to specific gap types, and a Remediation Planning Agent that ranks remediation actions from a remediation catalog via multi-signal scoring combining se- mantic similarity, regulatory alignment, and effort estimation. We evaluate this framework against four automatic compliance gap detec- tion baselines across 9 AI systems and 6 AI families. Our approach achieves a gap-type F1 of 0.831 (vs. 0.533 for the strongest baseline) and remediation P@5 of 0.872 (vs. 0.635 for retrieval methods), demonstrating promising potential for automated compliance workflows. The structured agent coordination reveals a path toward human-in-the-loop compliance systems where automated gap extraction reduces expert burden while maintaining auditability, and the taxonomy-driven design enables extension to emerging regulatory frameworks beyond