Large language models predict numbers better with meaningful variable names
Variable names provide priors: semantic grounding enables data-efficient numerical prediction by large language models
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
Predicting numbers from data can be hard when there isn’t much information available. The authors found that large language models (LLMs) do better at these predictions if the names of variables in the data match their meanings. This helps the models use what they learned before about the world to make smarter guesses. When new examples contradict prior knowledge, the models shift their predictions accordingly, similar to how people update beliefs when learning new facts. This method makes LLMs good at predicting numbers even with limited data.
What this means in practice
- •For data science teams: Design data systems using meaningful variable names to improve prediction accuracy when training data is limited.
- •For scientific computing teams: Use large language models to predict experimental or simulation outcomes in scientific domains with scarce data by leveraging variable name meaning.
Authors
Xin-Rui Ma, Shi-Zhi Chen, Hai-Yan Jiang, De-Cheng Feng, Michael Beer
Abstract
Large language models (LLMs) have achieved strong performance across diverse tasks, from question answering to code generation. Recent studies have found that LLMs can perform numerical prediction on tabular datasets via in-context learning. However, it remains unclear under what conditions and how LLMs use variable names and in-context examples for numerical prediction. Here we show that semantic grounding, the match between variable names and the values they describe, allows five LLMs without task-specific training to match or outperform seven supervised machine learning (ML) models across 20 datasets from nine scientific domains, particularly when examples are scarce. Reasoning traces indicate that variable names allow LLMs to draw on domain-specific knowledge acquired during pretraining. Using controlled tasks based on physical formulas, we further show that LLMs integrate domain-specific knowledge with in-context examples in a way consistent with Bayesian updating. Specifically, when the relationship between the inputs and the target cannot be inferred from the examples alone, LLMs rely on domain-specific knowledge as the prior. Conversely, as contradicting examples accumulate, LLM predictions shift progressively towards these examples, which act as the likelihood. Our work reveals how LLMs use domain-specific knowledge in numerical prediction and positions them as data-efficient predictors for data-scarce scientific domains.