Open Problems in AI Incident Governance
2026-07-06 • Computers and Society
Computers and SocietyArtificial Intelligence
AI summaryⓘ
The authors explain that AI systems can have problems after being used that weren't found before. To handle these problems well, there needs to be clear rules and systems for defining, tracking, and reporting AI issues. They looked at current rules and found that different groups use different ways to describe and report AI problems, making it hard to learn from these incidents properly. This inconsistency affects how well people can analyze and understand AI failures.
AI incident governanceincident reportingmonitoring practicestaxonomiesregulatory frameworksincident classificationsafety assessmentsAI failuresincident analysis
Authors
Harleen Kaur Sidhu, Rebecca Scholefield, Nour Annan, Kevin Hernandez, Isabel Nieh Hou, Abdulrahman Alshaikhi, Ze Shen Chin, Rokas Gipiškis
Abstract
AI systems may produce failures after deployment that pre-deployment safety assessments do not anticipate. Managing these failures requires what we refer to as adequate \textit{AI incident governance}, where having good definitions, taxonomies, monitoring practices, reporting mechanisms, and incident analysis is essential. We examine existing frameworks related to AI incident governance by regulatory bodies and independent efforts, and find that while there are frameworks that describe how individual functions can be performed, there is a lack of consistency within the aspects of definitions, classification, monitoring, and reporting. These inconsistencies apply to the types of incident data that is collected and reported, the ways in which they are categorised, and as a result, the depth, representativeness, and accuracy of analysis that can be performed.