Redundant checks in satellite data pipelines can share hidden errors
Independent Verification Paths Are Not Independent: A Case Study of Common-Mode Failure in a Satellite Catalogue Pipeline
Software EngineeringArtificial IntelligenceDatabases
Summary
This paper reveals that double-checking satellite data using two different computer methods does not always catch mistakes. The authors studied two ways of verifying satellite catalog numbers and found that both made the same wrong assumption, so all checks agreed even when the data were wrong. They traced the problem to shared misinterpretations and showed that even fixing the code did not fully solve it. This shows that some errors are about the meaning of data, which redundancy alone can miss.
What this means in practice
- •For data engineers: Design validation systems for satellite data that include source documentation checks to avoid shared semantic errors in redundancy.
- •For software quality teams: Improve quality assurance processes by acknowledging that independent implementation does not guarantee error independence when data interpretations are reused.
Authors
Fabio Rovai
Abstract
A common safeguard for a data pipeline is redundant computation: derive each published number by two routes built on different technology and refuse to exit when they disagree. We report one such gate failing, in a cross-catalogue integrity study of two open registers of Earth-orbiting objects. A gate comparing a set-based Python path with SPARQL queries over the emitted RDF graph printed ALL CROSS-CHECKS AGREE on seven counts. Three were wrong, one overstated more than fourfold (932 against 220). Both paths imported the same constants, which encoded a misreading of the source's status vocabulary, so the error was common-mode and the gate could not see it. We give the mechanism, an object-level ledger reconciling every figure, and three checks that go back to the source's documentation, measured on the defective code and on its correction. We then checked that correction against each object's phase history, held in a source file the pipeline never read. The correction was also wrong: 42 of its 261 disagreements are artefacts, and none of our three checks flagged them. Finally, in a controlled replication with three pinned models and tools disabled, 72 of 75 paths generated on request as independent checks computed the defective count, 29 of 30 even when the prompt carried the source's own definitions of the codes. The evidence is one pipeline and one defect family. Within it, redundancy verified implementation, and the errors that reached publication were errors of meaning.