Data-centric methods improve financial reasoning in language models
Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning
Computation and Language
Summary
Financial texts and questions are plentiful, but most aren’t ready for machines to learn financial reasoning directly. The authors created a step-by-step process that gathers and cleans financial training data with clear reasoning steps and verifiable answers. They also combine different training techniques to help language models keep and improve their financial skills. Their tests show that simple training can lower performance, but their advanced methods can raise it with fewer mistakes.
What this means in practice
- •For financial software developers: Build language models that better understand and reason about financial documents without losing existing knowledge.
- •For educational content creators: Generate improved financial question-answer pairs with clear reasoning steps for training or tutoring systems.
Authors
Zhirayr Hayrapetyan, Andrei Kalmykov, Denis Kokosinskii, Dmitry Stanishevskii, Dmitry Zmitrovich
Abstract
Financial text, textbooks, and question-answer pairs are abundant, but only a small fraction is directly usable for reasoning-focused post-training. Existing QA pairs often lack explicit reasoning, sufficient context, or reliably verifiable answers, while textbooks must first be transformed into synthetic training examples. We present a data-centric pipeline that constructs complementary corpora by mining open-source reasoning traces, distilling financial instruction data, and generating knowledge-graph-guided question-answer pairs from financial educational material. After semantic deduplication, three lightweight sequence classifiers select finance-relevant examples, reject under-specified questions, and identify tasks suitable for reinforcement learning with compact rule-based verifiers. For model adaptation, we study supervised fine-tuning and reinforcement learning, while self-distilled fine-tuning and post-training model merging are used to prevent the loss of financial capabilities already present in the starting model. We evaluate the adapted language models using FINESSE-Bench, reporting aggregate performance and changes relative to their starting checkpoints. Across the selected comparisons, ordinary SFT reduces FINESSE-Bench accuracy by 3.2-4.0 percentage points, whereas self-distilled SFT improves over the corresponding starting models by 1.0-2.8 points. Equal-weight merging recovers 3.0 points over its SFT parent and finishes 0.9 points above the original model; GRPO on hard tasks adds 0.4 points after self-distilled SFT or 3.0 points when applied directly to verifiable tasks. These results show that retention-aware adaptation can improve financial reasoning without the regressions observed after ordinary SFT.