Local mining improves distributed event rule detection with less communication

Mine Locally, Verify Globally: Exact Recovery, Compilation, and Distortion of Mined Temporal Rules

Distributed, Parallel, and Cluster ComputingLogic in Computer ScienceNetworking and Internet Architecture

Summary

When computers watch events happening on different machines, it’s tricky to combine what they see because clocks can be unsynchronized and some information might get lost. The authors studied how to collect and check patterns of these events efficiently by mining rules locally and then verifying them globally. This reduces the communication needed and still finds the important relationships between events. They also figured out when keeping track of timing details is important and how losing events affects detection accuracy.

What this means in practice

  • For network monitoring teams: Improve detection of network event patterns from multiple receivers while reducing communication costs and handling missing data.
  • For distributed system operators: Better aggregate event observations in systems with unsynchronized clocks to avoid errors from merging misaligned data streams.

Tested on one dataset.

Authors

Nikolaos Kekatos, Priyanka Sinha, Stylianos Basagiannis, Antonio Anastasio Bruto da Costa, Alexios Lekidis, Pabitra Mitra, Tom Nianios

Abstract

Specification mining extracts temporal rules that describe relationships between events in recorded system traces. Applying these rules in distributed monitoring systems is more difficult because observations come from different nodes, whose clocks may not be synchronized, and events may be lost or reordered during transmission. In this paper, we study how to aggregate the coverage of mined rules across observers, determine whether their temporal windows are needed during monitoring, and characterize their detection under event loss. Our evaluation uses IPv6 packet traces collected from 21 receivers across four continents. We show that local mining followed by global confirmation recovers centralized rule coverage under suitable conditions while substantially reducing communication overhead. In contrast, merging traces with unaligned clocks can distort coverage by introducing incorrect temporal matches. A window-gain criterion identifies rules that require temporal state, while our loss analysis and broker experiments demonstrate why contiguous losses can be more damaging than independent losses. The results identify separate requirements for aggregating observations, selecting monitor state, and handling event loss.