AI systems reshape their own data creating new mining challenges
Mining the Self-Rewriting System: A Vision for Data Mining on AI-Native Infrastructure
Machine Learning
Summary
Most data mining assumes that data is generated independently outside the AI system, but this paper explains that new AI setups change that assumption. The authors call these systems self-rewriting because the AI's own decisions affect future data and outcomes. They point out that analyzing individual data streams alone can be misleading since problems often arise from complex interactions across streams. The paper outlines key research challenges to better understand, monitor, and manage these evolving AI systems, using telecom networks as one example.
What this means in practice
- •For telecom ai engineers: Improve analysis of AI systems that adaptively modify their own logs and models in telecom networks to better diagnose complex failures.
- •For clinical decision support teams: Manage interactive AI tools in healthcare where the system’s own updates influence future training data and patient outcomes.
A position paper. It proposes an approach and reports no results.
Authors
Mayank Darbari, Chiman Salavati, Liang Wu, Dawei Wang, Jiong Zhang, Kelly Wan, Krishna Chakka, Liangjie Hong
Abstract
Classical data mining treats data as generated outside the learner. Emerging AI-native infrastructure invalidates this assumption: as agents observe system state, act, update retrieval stores and models, and promote new artifacts into the serving path, their own decisions reshape the future logs, traces, corpora, and evaluation records used for later training and release. We call this new research object a self-rewriting system and the resulting system-induced distribution shift endogenous drift. Our central, falsifiable claim is that cross-surface causes are not identifiable in general from marginal per-surface monitoring: different coupled failures can produce the same individual signals while differing in their joint behavior, requiring joint analysis for reliable attribution. This agenda comprises four foundational research problems: fusing heterogeneous drift signals into calibrated gating and attribution (C1); mining agent traces whose activity alphabet evolves (C2); maintaining knowledge fabrics through online reindexing, faithfulness, and compression decisions (C3); and safely promoting concurrent learning streams through one shared evaluation gate (C4). These remain fragmented across concept drift, performative prediction, process mining, streaming retrieval-augmented generation, and MLOps, without a unified data-mining agenda. We outline public benchmarks and evaluation criteria, using AI-native telecom only as a motivating exemplar with analogues in finance, software CI/CD, and clinical decision support.