Telemetry and Concealment in Self-Adapting Generative AI: Logging Architecture, Adversarial Model Hiding, and the Limits of Detection
2026-08-10 • Cryptography and Security
Cryptography and Security
AI summaryⓘ
The authors explain that traditional methods for managing risk in models assume models don’t change after being checked. However, new AI systems that update themselves while running break this idea, making old checks not enough. They propose a new system to track these changes both continuously and in steps, using special math tools and secure logging to keep records safe and clear. They also explore how a dishonest model creator might try to hide updates and suggest ways to catch or prevent such tricks. Overall, their approach helps keep self-updating AI models trustworthy without replacing the need for deeper audits.
Model Risk ManagementGenerative AISelf-adapting ModelsTelemetry ArchitectureMinimal Sufficient StatisticMerkle ChainIto FormulaKL DivergenceModel Hiding ProblemModel Validation
Authors
Sriram Nagaraj
Abstract
Model risk management (MRM) guidance assumes a static model lifecycle, in which models are developed, independently validated, and implemented without further autonomous modification. Continually self-adapting generative AI systems --- models that update their own weights during production deployment --- fundamentally violate this assumption and render point-in-time validation inadequate. This paper addresses the resulting governance problem in two parts. Part I develops a rigorous telemetry architecture for such models, operating simultaneously in discrete and continuous time. We establish a Minimal Sufficient Statistic for audit purposes, construct a tamper-evident Merkle chain for discrete weight sequences, derive the appropriate continuous-time generalization via the Ito formula, and propose event-driven logging via KL divergence stopping times that is both computationally tractable and meaningful for validation. Part II asks what happens when the model provider is adversarial. A firm deploying such a model has strong incentives to conceal learning updates that would trigger mandatory validation review. We formalize this as the Model Hiding Problem and provide a systematic taxonomy of six distinct attack strategies against the Part I architecture, spanning discrete and continuous time, with a formal countermeasure for each. Together the two parts establish a dual-regime architecture in which continuous telemetry is necessary but not sufficient, narrowing---but never replacing---periodic invasive audit. The framework is model-architecture-agnostic and is designed to satisfy the three pillars of traditional MRM.