Universal traceformer improves prediction of process events and timings
Universal Multi-Modal Traceformer: Integrating Heterogeneous Context for Process Event Prediction
Machine Learning
Summary
Many processes produce records of what happened and when, along with different types of supporting details like numbers or text. Existing models often ignore this extra information and just look at the sequence of events and times. The authors created a new model called Universal Multi-Modal Traceformer (UMT) that understands and uses these varied details to better predict what happens next and when. Tests on real-world data show UMT is more accurate than previous methods.
What this means in practice
- •For process analysts: Improve forecasting of next activities and timing in complex business process logs that include diverse data types.
- •For industrial automation teams: Enhance event prediction models for manufacturing or operational workflows by integrating heterogeneous contextual data beyond just timestamps and event types.
Authors
Fabian Spaeh, Jingxing Fang, Shandian Zhe, Bin Shen
Abstract
Event logs arise in a wide range of real-world processes, capturing not only event activities and timestamps but also multi-modal contextual information. Existing event-sequence models, including many temporal point process approaches, primarily model event activities and timestamps while overlooking heterogeneous context, such as numerical measurements, categorical attributes, textual descriptions, and metadata associated with individual events and entire traces. In this paper, we propose Universal Multi-Modal Traceformer (UMT), a unified framework for incorporating heterogeneous process context into next-event prediction. Built on a Transformer backbone, UMT introduces a universal feature encoder that maps diverse feature types into a shared representation space and handles contextual information at both the event and trace levels. UMT further develops a per-event Perceiver module that dynamically weights contextual features and adaptively integrates them into event-token representations. To accommodate the heavy-tailed and potentially multi-modal distribution of inter-arrival times, UMT represents each interval at multiple temporal scales and jointly predicts the corresponding scale-specific quantities. Experiments on 13 real-world event logs show that UMT improves both next-event activity and time prediction over existing approaches.