Architectural debt enforcement methods tested with partial information
Architectural Debt Enforcement under Incomplete Observation: A Controlled Study of Ratchets and Evidence Qualification
Software Engineering
Summary
Software projects often need to keep their internal structure clean, but sometimes the tools watching changes don't see everything fully. The authors study how to tell if a change is allowed or if there’s missing information making it unclear. They tested different methods across many files and scenarios to see which ones matched expected outcomes best, while acknowledging some unavoidable uncertainties remain. Their work focuses on understanding these limits rather than proposing new algorithms or tools.
What this means in practice
- •For software maintenance teams: Improve enforcement of architectural rules in large codebases where change observation is incomplete, reducing incorrect approvals of structural changes.
- •For software quality assurance teams: Use evidence qualification techniques to better explain uncertainty in software architecture enforcement diagnostics during build and integration.
Authors
Zhiheng Zhang
Abstract
Architectural debt gates must distinguish an allowed change from an incompletely observed one. We study a decision contract separating observation qualification, prohibited relation identity, inherited debt and authorized baseline updates. Across 240 fixed files from four repositories, the current observer and programmatic reference jointly qualify 166 files (69.17%), all with exact reference-relative edge agreement; 74 remain reference-UNKNOWN. Coverage varies from 13/60 to 59/60 by repository. On 480 constructed state scenarios sharing 60 source anchors, both the candidate ratchet and mature dependency-cruiser baseline match every expected outcome, whereas static and count-only baselines match 420 and 360. These results do not establish candidate superiority. Preserved failures include nine additional false-PASS probes and 120 unqualified full-tree timing attempts. A subsequent generic binding layer rejects 54 specified mismatches, accepts 54 current controls and permits legal repetition. Two bounded package scopes support 40 scenarios and 48 fresh-process functional checks; their costs remain tied to the measured implementations. Diagnostics explain residual uncertainty through type/runtime relations, installation context and generated-artifact provenance. TypeScript 5.9.3 and 6.0.2 yield identical outputs on ten paired emission witnesses, but neither that agreement nor verified release provenance changes the frozen UNKNOWN labels. The contribution is a controlled account of evidence-qualified enforcement and its limits, not language completeness, natural maintenance effectiveness or a new shrinking-baseline algorithm.