Root cause analysis improved by separating retrieval and reranking errors
Where Root Cause Analysis Fails: A Retrieval-Reranking Decomposition
Machine Learning
Summary
Finding the cause of problems in complex systems with many sensors is hard because existing methods mix up two different types of mistakes: not finding the real cause at all, and finding it but ranking it too low. The authors break down the problem into these two parts and test popular benchmarks, showing statistical approaches often rank the true cause poorly. They build a two-step system using a multi-signal retriever and a large language model (LLM) to rerank which matches or beats existing methods without needing complex causal graphs or labeled data. Adding simple system descriptions helps their method perform even better on all tested benchmarks.
What this means in practice
- •For industrial monitoring teams: Identify root causes of complex sensor anomalies more accurately using a two-stage retrieval and reranking approach without needing causal graphs or labeled data.
- •For infrastructure maintenance teams: Improve safety and reduce downtime by better pinpointing fault origins in monitored systems through improved analysis of sensor anomalies.
Authors
Hada Melino Muhammad, Luan Pham, Laure Barrière, Sachin Shetty, Leonardo Pulga, Flora D. Salim
Abstract
Identifying the root cause of an anomaly among hundreds of sensors is critical for preventing safety incidents and costly downtime in complex monitored systems. Existing studies evaluate root cause analysis (RCA) methods using top@k accuracy. We show that this metric has a fundamental blind spot: it conflates two failure modes, retrieval failure, where the true cause is never considered, and reranking failure, where it is considered but ranked too low. In this work, we introduce a retrieval-reranking decomposition and audit four well-known benchmarks to expose this blind spot. Our experiments show that, on benchmarks with complex faults, statistical baselines mis-rank the true cause 79-100% of the time, and graph-based methods never clearly beat the best statistical baseline, whether their causal graphs are learned on short fault windows, on retrieved candidate pools guaranteed to contain the cause, or on multi-day normal-operation data. Meanwhile, on simple benchmarks where faults manifest significantly at their origin, retrieval is nearly solved (98-100%). Guided by the decomposition, we build a two-stage pipeline combining a multi-signal retriever with an LLM reranker that, as one fixed configuration, matches or exceeds the best baseline's top@1 accuracy on all six benchmark suites (by up to +12 points), with no causal graph or labeled data required. When all methods rank the same retrieved candidates with the true cause guaranteed present, adding a short system-description document lets the reranker lead the best baseline by +7 to +18 points on every benchmark. Code is available at https://github.com/cruiseresearchgroup/DecompRCA.