Tabular foundation models learn relational data without training overhead
RelICL: Training-free Relational Learning with Tabular Foundation Models
Machine Learning
Summary
Learning from databases with multiple related tables is usually complicated and slows down when too many features are created. The authors found that a popular method called deep feature synthesis, which merges related tables into one big table, faces problems with too many features and missing interactions between features. They introduced a new method called RelICL that passes information through the tables step-by-step, reducing feature explosion and capturing interactions better. Their tests showed that RelICL performs as well as the best existing methods without needing extra training.
What this means in practice
- •For database engineers: Improve relational database predictions without retraining models by using stepwise schema graph information propagation.
- •For business intelligence teams: Generate efficient and accurate features from complex relational data for analytics without extensive feature engineering.
Authors
Simon Forbat, Rainer Gemulla
Abstract
Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens a relational schema into a single table by adding aggregates of the other tables' columns as features. This approach is appealing because it directly benefits from improvements to or customization of the underlying tabular foundation model. In this paper, we identify two key problems with DFS: feature explosion and interaction blindness. The first problem arises because the number of DFS features grows quickly as the schema becomes more complex, limiting scalability and performance. The second problem arises because column-wise aggregates do not account for feature interactions, limiting performance. We propose and explore an alternative method termed RelICL, which keeps the benefits of DFS but alleviates these two problems. At its heart, RelICL propagates and fuses information step by step through the schema graph, using the same tabular foundation model that is eventually used for prediction to do so. In our experimental study using RelBench tasks, RelICL was on par with the strongest approach based on deep feature synthesis.