Tool augmented systems improve time series question answering accuracy
T-SMART: Mechanism-Level Attribution for Tool-Augmented Time-Series Question Answering
Machine Learning
Summary
Large language models often find it hard to answer questions involving time-series data because they have to handle numbers as text and do calculations. The authors created T-SMART, which splits the work: the language model figures out what question is being asked and picks operations, while separate tools handle the actual math. This approach improved accuracy by a large margin, showing that relying on precise calculation tools is more important than extra language reasoning for these tasks. Small added benefits come from better language understanding and recognizing relevant data features.
What this means in practice
- •For data engineers: Use deterministic tools alongside language models to improve accuracy in automated time-series querying systems.
- •For business analysts: Deploy tool-augmented question answering systems to obtain more accurate numerical insights from serialized time-series reports.
Authors
Ivan Delgado, Himansi Gupta, Bishal Khatri, Niharika Sapre, Lameta Shamoon, Onat Gungor, Tajana Rosing
Abstract
Large language models (LLMs) can struggle with time-series question answering (TS-QA), especially when numerical signals are serialized as text and require explicit computation. Tool-augmented approaches improve performance, but existing systems often intertwine language reasoning, computation, and perception, making it difficult to determine which components drive the gains. We present T-SMART, a neurosymbolic framework that separates these roles: a frozen LLM interprets questions and selects operations, deterministic tools perform numerical computation, and structured perception is invoked only when needed. Controlled paired ablations show that deterministic computation provides the dominant benefit, improving accuracy by 31.7 percentage points over direct LLM reasoning on serialized time series, while language understanding and perception offer smaller complementary gains. These results indicate that tool-augmented TS-QA benefits primarily from reliable numerical execution rather than additional language-model reasoning and provide a controlled framework for analyzing component contributions in neurosymbolic time-series systems.