Task aligned pretraining improves transaction event predictions
TAPES: Task-Aligned Joint Predictive Pretraining for Transaction Event Sequences
Machine LearningArtificial Intelligence
Summary
Predicting different outcomes from sequences of transaction records is challenging because each prediction type depends on different aspects of the data. The authors propose TAPES, a method that selects the best set of prediction targets by trying short training runs and then jointly trains a model using a shared hidden variable. This approach better captures relationships between different prediction targets and improves performance on various financial prediction tasks. The authors tested TAPES on real-world and public financial datasets, showing it works better than previous methods.
What this means in practice
- •For financial risk teams: Improve risk prediction models by pretraining on task-aligned transaction targets that capture multiple prediction semantics jointly.
- •For online commerce analysts: Enhance customer behavior and transaction outcome predictions by selecting optimized target sets for sequence models.
Authors
Ruobing Zuo, Weiyu Han, Huilin He, Haoyu Jiang, Ze Zhao, Jie Sheng, Xiao Tan, Shuai Chen, Tianyi Zhang
Abstract
Transaction event sequences support diverse prediction tasks in online commerce and digital financial services. The same transaction records can yield prediction targets with different semantics, including values, changes, and relations to past events. These targets impose different constraints on history representations, but their predictability alone does not establish the downstream utility of a target set. Moreover, target-wise prediction losses do not explicitly model conditional co-occurrence among targets. To address these challenges, we propose TAPES, a task-aligned joint predictive pretraining framework for transaction sequences. TAPES constructs a finite collection of candidate target sets from admissible field-operator combinations. It selects a target set according to the downstream utility of representations learned through short joint pretraining trials under matched budgets. It then reinitializes the encoder and pretrains it on the selected targets through a joint likelihood parameterized by a shared discrete latent variable, before downstream supervised adaptation. Experiments in a real-world transaction-risk setting and on multiple public financial benchmarks spanning diverse downstream tasks demonstrate that TAPES outperforms representative baselines. Ablation studies further validate the contributions of predictive target selection and joint modeling.