Evolving language model features improves interpretable classification accuracy
Evolving LLM-Generated Features for Interpretable Classification
Machine Learning
Summary
Large language models (LLMs) can make decisions but often do so in ways that are hard to understand, which is a problem in areas like medicine or finance. The authors propose a method that uses an LLM to create simple, clear rules or features that represent its decisions in natural language. These features are refined over multiple rounds to improve accuracy and fairness. This approach makes the decisions easier to check and explain, especially in sensitive areas like credit risk assessment. Their tests show this method works better than just asking the LLM once or directly using it to classify, especially when the LLM struggles with domain knowledge.
What this means in practice
- •For credit risk teams: Produce transparent credit scoring models using evolved language-based features for better auditability and balanced predictions.
- •For healthcare data teams: Create interpretable diagnostic classifiers with natural language features that reveal decision logic for compliance and trust.
Authors
Jack Butler, Zainab Afolabi, Nikita Kozodoi
Abstract
Large language models (LLMs) are increasingly used as classifiers, yet they operate as opaque systems whose decisions are difficult to interpret, which complicates their use in regulated domains such as credit scoring or medical diagnosis. We propose an evolutionary framework that iteratively discovers natural language feature definitions (rubrics) for interpretable classification. An LLM generates candidate binary features, evaluates each sample against them, and the resulting vectors can be used to train a transparent classifier such as logistic regression. The feature set evolves over multiple iterations guided by classification errors, per-class activation rates, and feature ablation scores. We evaluate across three benchmarks, including a credit risk dataset representative of regulated domains, comparing single-shot LLM rubrics, evolved rubrics, and direct zero-shot LLM classification. Evolved features improve over single-shot rubrics by +2.9 pp on average and outperform zero-shot LLM classification on two of three tasks, while providing fully auditable decision logic. On the credit risk task, the zero-shot LLM performs at chance (50.7%) with a strong bias toward a single class, whereas evolved features achieve balanced, interpretable predictions. Crucially, this failure is invisible in aggregate accuracy and surfaces only under per-class auditing. Analysis reveals that evolution is most effective when label boundaries cannot be inferred from category names alone or when the LLM lacks reliable domain-specific reasoning.