Budgetverify improves efficiency in financial question answering verification
BudgetVerify: Budget-Tiered Verification for Financial QA
Artificial IntelligenceMachine Learning
Summary
Answering financial questions accurately often means checking if numbers and calculations are right, which can take a lot of computer power. The authors propose BudgetVerify, a system that decides how much checking is needed for each answer, sometimes skipping checks and other times doing detailed verification. This approach uses information about the question and the generated answer to save computing resources while keeping answers accurate. The system was tested with different AI models and consistently balanced accuracy and cost better than using the same checking method every time.
What this means in practice
- •For financial technology teams: Reduce computing costs in verifying financial AI answers by selectively applying costly checks only when needed.
- •For enterprise ai platform operators: Optimize resource allocation in automated question answering services by dynamically adjusting verification efforts based on question and answer features.
Authors
Janet Jenq, Hongda Shen
Abstract
Financial question answering often requires precise numerical extraction, unit handling, and arithmetic over tables and text, but applying expensive verification uniformly wastes test-time compute. We propose BudgetVerify, a budget-tiered generator-verifier framework that routes each generated answer to one of three verification tiers: no verification, lightweight check-and-revise, or higher-cost solve-first-then-compare verification. The router is trained from offline correctness and token-cost outcomes and, at test time, selects a verification tier using information available before verification, including the question, context statistics, the generated answer, and associated generator metadata. The selected tier either returns the generated answer directly or invokes the corresponding verifier. Across six commercial and open-weight base models, BudgetVerify consistently produces more efficient accuracy-cost Pareto frontiers than fixed verification policies by selectively allocating stronger verification only when it is useful. Although absolute performance varies across models, these efficiency gains and the resulting qualitative frontier shape are consistent across generator models.