Papers for

public transit managers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Language models choose cost-effective forecasts to improve decisions

Forecast Workflow Bench: Evaluating Language-Model Decisions with Budgeted Forecast Tools

Abstract: Time-series foundation models (TSFMs) provide forecasts for operational decisions, but accuracy alone does not determine their value. Evaluating agents that use these models requires measuring decision quality and forecast cost. FWBench evaluates this capability on 1,251 electricity and cycle-hire cases using fixed forecast tools and simulated capacity contracts. Agents select models, histories and horizons, then submit capacities to minimize a stated loss-cost objective. We evaluated two hosted and eight local configurations, including small language models, and tested local models with and without TSFMs. GPT-6 Astra bought inexpensive short-horizon forecasts selectively, using 2.5% of the budget, and outperformed fixed policies when the saved decisions were scored with three loss-cost weightings. FWBench enables reproducible evaluation of how language models select and use time-series forecasts to make decisions under cost constraints.

Wed 23 SeptMachine LearningArtificial Intelligence

The gist

Making good predictions for things like electricity use is important but using every forecast can be expensive. This paper presents FWBench, a tool that tests how well language models decide which forecasts to buy and use under budget limits. The authors show that some language models can pick cheaper, short-term forecasts and still make better decisions than fixed methods. This helps understand how language models balance forecast costs and benefits in real-world decisions.

Open → 2609.27385v1