System fixes broken traces across video platform microservices
Backstitch: Restoring Request Causality Across a Production Microservice Fleet
Distributed, Parallel, and Cluster Computing
Summary
Many small services handle video requests, passing information along so the whole process is understood. Sometimes the connection breaks silently, causing loss of tracking without failing the request. The authors created Backstitch, a system that finds where these breaks happen and repairs them automatically by analyzing and replaying the service calls. This restores the tracking without changing the actual work, cutting errors in half over months.
What this means in practice
- •For software reliability engineers: Automatically detect and repair missing request context flows in complex microservice architectures to improve traceability.
- •For cloud service operators: Maintain accurate causal tracking across asynchronous messaging and callbacks in large-scale service platforms.
- •For enterprise middleware developers: Enhance middleware platforms by integrating context restoration to preserve request tracing in heterogeneous environments.$Commercial implications: Enables middleware providers to offer built-in transparent trace continuity features for clients with microservice ecosystems.
Authors
Ziyue Dang, Qiuyu Wu, Haoyun Xu, Tongjue Wang, Yongqing Ling, Weihao Chen, Guangming Luo
Abstract
A major video platform runs on thousands of microservices, each request propagating a context so downstream work can be traced and governed. At handoffs outside instrumented paths, e.g., custom queues and callbacks, the payload continues but the context does not, and the request still succeeds under existing tests. Such breaks are silent and widespread: 673 of 1,133 services carried at least one. Backstitch, a specialized agentic system, repairs them using the surviving execution as its reference: replay determines whether a suspicious call is request-correlated, source analysis reaches the responsible handoff, a bounded change restores its contract, and the same replay validates the fix. Repairs restore the causal chain without disturbing the work it describes: breaks at 240 of the repaired calls fell from 90.46% to 4.69%, and over 112 days the fleet's break rate more than halved.