D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring
Machine Learning
Summary
The authors studied how to predict the future steps and time left in business processes, especially when data is limited or unpredictable. They introduced a method called D-TAIA that uses a small, efficient large language model tuned with special training and retrieval techniques to handle these tough conditions better. Their experiments on real datasets showed that D-TAIA performs as well as or better than existing models. They also found that ideas from language and vision processing can help in process monitoring, although different parts of their method matter more depending on how complex the data is.
Predictive Process MonitoringLarge Language ModelsDomain-aware TrainingTriplet LossFAISSSequential ReasoningFine-tuningDistributional ShiftProcess EntropyRemaining Time Prediction
Authors
Sjoerd van Straten, Christine Jacob, Marwan Hassani
Abstract
Predictive Process Monitoring (PPM) enables organizations to forecast future process behavior, such as the next activity and remaining time of ongoing cases. In practice, three conditions cause existing methods to degrade, namely data scarcity, high process entropy and distributional shift. While Foundation Models (FMs), especially Large Language Models (LLMs), offer a new paradigm through broad sequential reasoning, adapting them to multi-task PPM under these conditions remains an open challenge. Existing FM-based approaches either lack mechanisms for handling distributional shift or rely on direct regression heads that can be structurally misaligned with continuous time prediction tasks. This paper introduces D-TAIA (Domain-aware Training and Attention-based Inference Architecture), a framework for a joint next activity and remaining time prediction task via parameter-efficient fine-tuning of an FM backbone. Our approach combines domain-aware triplet loss (DATL) pre-training with FAISS-based nearest neighbor retrieval for remaining time prediction, and adopts the TAIA inference strategy to preserve pre-trained sequential reasoning during fine-tuning. Evaluated across four real-world event logs, D-TAIA consistently shows SOTA or competitive performance compared to a fine-tuned LLM and a recurrent neural network baseline. Ablation studies confirm that techniques from NLP and computer vision can be transferred effectively to PPM with only a 10M-parameter backbone, though component contributions vary by dataset entropy.