Summary
Railway systems have strict safety rules, so AI is only used for non-critical tasks today. The authors look at three key areas—robustness, operational limits, and explainability—to help AI systems gain trust and meet strict regulations. Robustness means the AI works well under all conditions, Operational Design Domain defines where the AI can safely operate, and explainability helps humans understand AI decisions. These ideas are already studied outside railways and can be adapted to railway needs. Building trust across these areas helps AI get accepted and used safely in mission-critical railway tasks.
What this means in practice
- •For railway system engineers: Implement AI systems that meet safety regulations by verifying robustness, defining operation conditions, and improving explainability to gain regulator trust.
- •For industrial safety regulators: Develop frameworks that assess AI tools based on robustness, operation domains, and explainability to authorize safe use in critical railway operations.
A position paper. It proposes an approach and reports no results.
Authors
Lefebvre Renard Clément, Lébé Vincent, Da Silva Ribeiro Pereira Ricardo, Sundell Johan, Jaoul Arnaud Saiah Kenza, Mijatovíc Nenad
Abstract
Artificial Intelligence (AI) is currently only applied to non-safety critical applications due to the strict standards and regulations for railway industries. We propose to review the three main fields necessary to increase trust in data science and AI algorithms and reach compliance: robustness, Operational Design Domain (ODD), and explainability. Robustness is the ability of an AI system to maintain its level of performance under any circumstances (ISO24029). ODDs allow the explicit definition of operating conditions under which a system is intended to operate, according to the recently published DIN DKE SPEC 99004. Explainability is the property of an AI system to express important factors influencing the AI system results in a way that humans can understand. Those 3 domains of research are already well investigated by nonrailway actors, with algorithms and methods ready to use for railway applications. A system view is necessary to ensure all trustworthy requirements interact continuously in a safe MLOps environment thereby fostering acceptance from regulators, operators and the public. Beyond safeguarding safety-critical applications, we aim to show that fostering deep trust in AI, as now required by regulatory frameworks worldwide, will unlock its full potential and transform the pace of adoption across mission-critical domains.