Tabular data predictions improved without labeled examples using language models
Marginal Response Surface Elicitation for Zero-Label Tabular Learning
Computation and Language
Summary
Predicting outcomes from structured data usually needs labeled examples, which can be hard to get. The authors propose a way to use large language models to guess how features relate to outcomes without any labels. Their method, called MARS, asks the model about representative feature values, combines its answers carefully, and makes predictions without further queries. This approach performed better than simply asking the model directly and was more efficient across several test tasks.
What this means in practice
- •For data science teams: Build predictive models on structured datasets without needing labeled training data by converting LLM insights into tabular classifiers.
- •For healthcare analytics teams: Generate reliable predictions for clinical tabular data where labeled examples are scarce using elicited knowledge from language models.
Authors
Liangyu Teng, Yicheng Ding, Jing Liu, Hengsong Liu, Juncen Guo, Hongru Li, Jingyu Zhang, Liang Song
Abstract
Tabular learning uses structured data to predict target outcomes. Traditionally, this process has relied on labeled data. However, large language models (LLMs) can be used to elicit domain priors based on the task description and feature semantics, thereby enabling predictions without labeled data. We propose Marginal Response Surface Elicitation (MARS), a method that transforms feature-level LLM priors into a reusable, zero-shot tabular classifier. To construct this classifier, MARS selects representative values for each feature from unlabeled data and prompts the LLM to provide corresponding class support scores and feature weights. It then aggregates multiple responses using the median to construct feature response functions, and makes predictions through their weighted sum without further LLM queries. Across eight tabular benchmark tasks, MARS achieves the highest average AUC and AP, outperforming direct prompting by 1.97 and 6.21 percentage points respectively, while substantially reducing end-to-end costs. Evaluations with LLMs of different sizes further demonstrate its predictive advantage over direct prompting.