Tabfm enables zero-shot predictions for tabular data without tuning
TabFM: A Zero-Shot Foundation Model for Tabular Data
Machine Learning
Summary
Making predictions from tables of data usually requires building a model from scratch for each new task, which can take time and effort. The authors show a large model called TabFM that can predict outcomes for new tables without any extra training by learning from many simulated tables. This means it can quickly give good predictions on many different datasets without tuning. Extensions of TabFM improve its accuracy further using feature expansions and guidance from large language models. Across many tests, TabFM even beats standard automated methods that are tuned for each dataset.
What this means in practice
- •For data science teams: Quickly apply a single pretrained model to new tabular datasets for prediction without extra tuning or training steps.
- •For business analytics teams: Generate automated, accurate predictions from business tables by combining pretrained tabular models with language-based feature engineering.$Commercial implications: Enables building commercial analytics tools that provide accurate, ready-to-use predictive insights from business tables without manual model building.
Authors
Weihao Kong, Erez Louidor Ilan, Shuxin Nie, Taman Narayan, Rajat Sen, Yichen Zhou, Deqing Fu, Samet Oymak, Abhimanyu Das
Abstract
Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. TabFM produces calibrated zero-shot predictions in a single forward pass without task-specific tuning. Trained entirely on synthetic tables generated from structural causal models, TabFM learns general tabular representations that transfer zero-shot to real-world tasks. Across all 51 benchmark datasets in TabArena (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines. Two extensions over the same frozen weights improve performance further on both tracks: multi-view feature expansion with ensembling and post-hoc calibration (TabFM+), and LLM-guided, dataset-specific data processing and feature engineering (TabFM-Auto).