Machine learning methods compared on biomedical data with many variables
TabBench-Bio: A Living Benchmark for Machine Learning on High-Dimensional Biomedical Tables
Machine LearningArtificial Intelligence
Summary
Many biomedical studies collect thousands of measurements but only have a small number of labeled examples to learn from, making it hard for computers to accurately analyze the data. The researchers created TabBench-Bio, a growing collection of 43 biomedical datasets to test and compare how well different machine learning methods work on these challenging tables. They found that certain advanced models called tabular foundation models generally perform best, though which method is best changes depending on the type of data. They also made their results easy to reproduce and invited the community to add more datasets to keep improving the benchmark.
biomedical tablesmachine learningtabular datacross-validationfoundation modelslogistic regressionbootstrapAutoMLbenchmarkdatasets
Authors
Jules Kreuer, Sofiane Ouaari, Julia Hellmig, Julius Braitinger, Nico Pfeifer
Abstract
Biomedical tables often combine thousands of measured variables with only tens or hundreds of labelled samples, a regime that is poorly represented in general-purpose tabular benchmarks. We introduce TabBench-Bio, a living and interactive benchmark of 43 biomedical datasets spanning multiple domains. Under a shared cross-validation protocol, we compare classical estimators, neural networks, and tabular foundation models across 28 feature-by-sample operating points. At the reference cell of 10,000 features and 100 training samples, RealTabPFN v2.5 has the highest point estimate, followed by Logistic Regression and TabDPT, whose point estimates are nearly identical. A paired bootstrap over the target pool separates RealTabPFN v2.5 from Logistic Regression by 145 Elo (95% interval [59, 232]). Tabular foundation models generally occupy the leading ranks, while the strongest configuration depends on the operating point and biomedical modality. The AutoML framework AutoGluon, using its one-hour "extreme" preset, is configured as a separate resource-intensive reference and is reported here at the reference cell. Fold-level predictions, run status, and deterministic aggregations make every reported result reproducible and reusable. We invite the community to contribute: TabBench-Bio is designed to grow, and we welcome submissions of new biomedical tabular datasets, particularly from underrepresented assays and clinical endpoints, for inclusion in future releases. The interactive leaderboard is available at: https://tabbench-bio.eu