Source-Bounded Exact Recovery over Docker's Logs API
2026-08-03 • Distributed, Parallel, and Cluster Computing
Distributed, Parallel, and Cluster ComputingOperating SystemsSoftware Engineering
AI summaryⓘ
The authors studied how Docker logs can be reliably recovered after interruptions like downtime or restarts. They found that simply saving a reading position isn’t enough to guarantee exact recovery of all log records. Their approach involves tracking records with more detail, allowing them to detect and fix missed or duplicated logs. When tested, their improved method in LogDeck reliably recovered all logs, unlike a similar tool which missed some during source restarts or removals. Overall, they show that properly handling Docker container lifecycles is key to accurate log recovery, not just storing where you left off reading.
Docker LogsLog RecoveryPersistenceLog APICollectorRead PositionLifecycle ReacquisitionData ExactnessMultiset OracleDaemon Restart
Authors
Kelvin Amoaba
Abstract
Docker can retain records that a collector misses before attachment or during downtime. A persisted read position does not by itself ensure recovery after lifecycle changes. We study what exact recovery contract is achievable through Docker's supported Logs API. We define source-bounded exactness: every retained, distinguishable source record eventually appears exactly once in durable collector output. Our method uses a generation-aware multiset oracle that separates source truncation from collector omission and exposes simultaneous loss and replay. Applied to LogDeck, it uncovered a start-to-attachment race; a one-record attachment overlap, finite Docker-API reconciliation, and exact insertion closed the tested boundary. We compare the fixed revision with unmodified Grafana Alloy 1.18.0, which uses the same API and persists read positions; across 120 collector-runs, LogDeck was exact in 60/60 and Alloy in 20/60. Alloy succeeded at guarded startup and process pause but omitted retained history when recovery required discovering an exited or restarted source. In a causal control, a 5,000-record source exited before collection: stock discovery was exact in 0/20 trials and acquired nothing, while the same reader given the container ID recovered all records exactly in 20/20. This reproduced on OrbStack and independent Ubuntu hosts with Docker 29.4.0 and 24.0.9; both collectors recovered daemon restart, while neither recovered records after source removal. Exactness assumes distinct tuples of physical generation, timestamp, stream, and bytes; 200,000 byte-identical records across two drivers produced no observed collisions. Our results show that lifecycle reacquisition, not a persisted position alone, determines exact recovery within the retained-source horizon. This is a bounded interface claim, not a universal collector ranking or proof of collision freedom.