Llm guided offline feature engineering transforms long text for risk prediction
Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Search
Machine LearningComputation and Language
Summary
Many risk prediction systems use structured data but miss valuable clues hidden in long text descriptions. Processing long text in real time with large language models (LLMs) is slow and expensive. The authors present a way to use LLMs ahead of time to create programs that extract useful features from text quickly during actual use. Their method searches for feature programs by piecing together code blocks and avoids getting stuck on poor choices by rolling back and exploring multiple paths. This approach improves prediction accuracy and works in real-world financial risk tasks.
What this means in practice
- •For financial risk analysts: Create efficient offline feature extraction from long textual data for improved financial risk prediction models.
- •For enterprise data teams: Deploy feature programs that transform unstructured long text into structured features without needing real-time LLM calls.
Authors
Ziming Dai, Dabiao Ma, Ziheng Guo, Jack Dong, Zimu Zhou
Abstract
Industrial risk-control systems typically rely on structured-data models for efficient prediction, yet substantial valuable information remains embedded in unstructured long text. Extracting this information through manual feature engineering is labor-intensive, while requiring a large language model (LLM) to process every real-time input may not meet practical deployment requirements. To address this challenge, we propose LLM-BlockFE, an LLM-guided offline feature construction framework that converts long text into executable feature programs, thereby avoiding LLM calls during online inference. LLM-BlockFE constructs feature programs by incrementally appending immutable code blocks and evaluates candidate features using a downstream model. To address the tendency of conventional greedy search to become trapped in suboptimal solutions, our method introduces a block-level rollback mechanism based on depth-calibrated credit allocation and advances multiple independent search trajectories in an interleaved manner, reducing redundant exploration by sharing fixed descriptions of each trajectory's exploration direction. After the search, the resulting programs are frozen and deployed to extract structured features for downstream prediction models. Across two public and two private datasets, LLM-BlockFE achieves absolute AUC improvements of 0.0069 to 0.0358 over the strongest baseline on each dataset in the full-dataset comparison. Post-launch monitoring across five deployed financial risk-control applications shows absolute KS improvements of 0.02 to 1.56 percentage points over the existing manually designed strategy.