AI agent runs recorded in sealed, replayable, auditable evidence files
NovaFabric: A Black-Box Recorder for AI Agents with Sealed, Replayable, Diffable and Auditable Runs
Distributed, Parallel, and Cluster ComputingCryptography and SecuritySoftware Engineering
Summary
AI agents do many things during a run, like making calls and using tools, and tracking what happens is complicated. Existing logs are incomplete and can be changed, so they don't reliably show what happened. The authors created NovaFabric, a tool that captures AI runs as fixed, replayable files that can be checked and compared later. This makes investigating or auditing AI behavior easier and more trustworthy.
What this means in practice
- •For ai platform operators: Maintain sealed, replayable logs of AI agent executions to support trustworthy audits and failure investigations.
- •For cloud orchestration teams: Track and audit complex job runs over distributed systems like Kubernetes or Slurm using hierarchical run capsules.
- •For security compliance teams: Use cryptographically signed export bundles for verifying AI system runs to meet regulatory or internal compliance needs.$Commercial implications: Enables selling tamper-proof audit tools for AI systems to compliance teams requiring digital evidence.
Authors
Mohsen Seyedkazemi Ardebili
Abstract
An AI agent run is no longer a function call. It is a cascade of prompts, model calls, tool invocations, file and network effects and sub-agents. When such a run misbehaves, or an auditor asks what happened, logs and traces answer poorly: they are mutable, partial and cannot be re-executed. This technical report describes NovaFabric, an open-source (Apache-2.0) black-box recorder for AI agents that turns a run into a sealed, replayable, diffable and auditable evidence capsule. The design rests on two formats only. The Run Capsule is a directory of flat files and the source of truth; every database is a rebuildable index over capsules. The Evidence Bundle is the signed export for an auditor. Everything else follows existing standards: OpenTelemetry GenAI, OpenLineage, W3C PROV, in-toto/DSSE, RFC 3161 and CycloneDX. A single-node run is the smallest case of a distributed one; the schema supports a capsule hierarchy for Slurm arrays and Kubernetes jobs, though automatic parent finalization is library-only today. The report covers the architecture and the engines behind the verb chain capture, seal, replay, diff and audit: fail-closed mocked replay; a diff gate with a stable exit-code contract; DSSE signatures that bind their intent and a capsule root that commits to file names; a capsule treated as untrusted input; signer identity-trust levels with offline TSA and CRL checking; maker-checker promotion against an operator-held trust list; and a deny-by-default authorization table for every dashboard route. It presents no new experiments; the evaluation is in arXiv:2609.12582. Every count except the private ADR tally is generated by a probe script run on the public source at the v0.107.0 tag (150 top-level commands, 84 Python subpackages, 16,721 test functions), every capability carries an explicit maturity label, and honest-limits boxes state where the system stops.