Machine learning benchmark compares models on biomedical tables
TabBench-Bio: A Living Benchmark for Machine Learning on High-Dimensional Biomedical Tables
Machine LearningArtificial Intelligence
Summary
Biomedical data tables often have many features but only a few samples, making it hard to test machine learning methods fairly. The authors created TabBench-Bio, a growing collection of 43 biomedical datasets to compare different machine learning methods under consistent conditions. They found that specialized tabular models generally perform best, but the top method depends on the data type and sizes. This benchmark is interactive and invites others to add datasets to better cover underrepresented biomedical areas.
What this means in practice
- •For hospital data teams: Evaluate and select machine learning models suitable for biomedical datasets with many features but few samples using a shared benchmark.
- •For clinical software developers: Integrate high-performing tabular foundation models into biomedical analysis tools informed by benchmark comparisons on diverse biomedical tables.$Commercial implications: Enables development of software with improved predictive accuracy for clinical decision-making based on validated model performance across biomedical data.
Authors
Jules Kreuer, Sofiane Ouaari, Julia Hellmig, Julius Braitinger, Nico Pfeifer
Abstract
Biomedical tables often combine thousands of measured variables with only tens or hundreds of labelled samples, a regime that is poorly represented in general-purpose tabular benchmarks. We introduce TabBench-Bio, a living and interactive benchmark of 43 biomedical datasets spanning multiple domains. Under a shared cross-validation protocol, we compare classical estimators, neural networks, and tabular foundation models across 28 feature-by-sample operating points. At the reference cell of 10,000 features and 100 training samples, RealTabPFN v2.5 has the highest point estimate, followed by Logistic Regression and TabDPT, whose point estimates are nearly identical. A paired bootstrap over the target pool separates RealTabPFN v2.5 from Logistic Regression by 145 Elo (95% interval [59, 232]). Tabular foundation models generally occupy the leading ranks, while the strongest configuration depends on the operating point and biomedical modality. The AutoML framework AutoGluon, using its one-hour "extreme" preset, is configured as a separate resource-intensive reference and is reported here at the reference cell. Fold-level predictions, run status, and deterministic aggregations make every reported result reproducible and reusable. We invite the community to contribute: TabBench-Bio is designed to grow, and we welcome submissions of new biomedical tabular datasets, particularly from underrepresented assays and clinical endpoints, for inclusion in future releases. The interactive leaderboard is available at: https://tabbench-bio.eu