Papers for

business data analysts

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Human in the loop AI cuts document annotation time by up to 80 percent

GAIDA: Generative AI for Human-in-the-Loop Document Annotation

Abstract: Document annotation is a fundamental yet labor-intensive step in building document information extraction systems. While Optical Character Recognition (OCR) and Large Language Models (LLMs) can automate parts of this process, we argue that annotation systems should be judged not by how completely they remove human effort, but by how effectively they redirect it toward review and correction. We propose a modular human-in-the-loop architecture that integrates OCR-based extraction, LLM-based semantic pre-annotation, and human validation in a single editable workflow, and present \textbf{GAIDA}, a research prototype serving as its reference implementation. GAIDA supports interchangeable OCR engines, LLM backends, and prompt configurations, and enables stage-specific measurement of annotation effort. In an initial single-annotator, stage-specific study on the FUNSD scanned-form and WildReceipt retail-receipt benchmarks, OCR assistance reduced region-creation and transcription time by up to \textbf{80.9\%}, and reviewing LLM pre-annotations reduced semantic-labeling time by up to \textbf{73.7\%}, despite imperfect pre-annotations of 78.86\% and 86.98\% Micro F1 on FUNSD and WildReceipt, respectively. Finally, we reuse the same implementation, adapting only OCR and prompt configuration, to build and openly release \textbf{AraReceipt}, a manually annotated dataset of 100 Arabic receipt images labeled with 25 key-information classes. The results indicate that imperfect automated outputs provide practical value when presented as editable suggestions under human control.

Mon 19 OctArtificial Intelligence
The gist
Annotating documents by hand is slow and tiring. The authors designed a system called GAIDA that combines automated text reading with AI-generated suggestions that humans can easily fix or approve. This approach doesn’t try to remove humans but rather lets them work faster by reviewing and correcting these suggestions. Tests showed it reduced some tasks by over 80%, even though the AI wasn’t perfect. They also used the system to create a new Arabic receipt dataset by swapping out components easily.
Open → 2610.26362v1

Self improving agent grows language for exact question answering

A Self-Improving Agent that Grows its Own DSL: Oracle-Free Operator Induction for Exact Question Answering

Abstract: A self-improving agent has to decide which new skills to keep, and in practice it keeps whatever earns a high reward, convinces a judge model, or compresses its library, even when the skill is wrong. That is fine until the answers have to be exactly right: over a table of transactions, a total is either right or wrong. We ask a harder question: can an agent grow what it can compute with no reward, no judge, and no answer key, and still be exactly right? A single frozen model runs two nested loops over a small, exact domain-specific language (DSL). The inner loop answers new questions only when two independently written expressions agree on every record. The outer loop discovers its own missing capability and authors an operator for it, admitted only when an independent second implementation agrees and the result obeys a conservation law, all without ground truth. The inner loop roughly triples the answerable-question set, and the outer loop self-extends on disjoint starting sets, inventing operators outside the starting closure; a no-induction control answers none of the new questions. The admission rule is empirically sound: it rejects 138 of 138 buggy operator variants under an independent reference, even one written by a different model family, where the admit-if-it-runs, self-consistency, and compression signals the field relies on admit most of them. Unmodified, the same mechanism self-extends on two further domains with unrelated schemas, business-ledger and clinical-encounter question answering, growing their answerable sets more than tenfold and nearly twentyfold. The loop transfers across model families too: run with further proposers, the gate stays sound and independent families self-extend comparably, with only the induction yield tracking each proposer's DSL fluency.

Wed 14 OctArtificial IntelligenceComputation and Language
The gist
Some AI systems learn by trying new skills and keeping those that seem useful, but they can still make mistakes in exact tasks like math or data totals. The authors created an agent that teaches itself new ways to solve exact questions without needing answers or judging systems. It only accepts new methods when two independent computations agree perfectly and follow strict rules, helping it avoid errors. This approach works across different types of data and tasks, increasing the number of questions it can answer exactly and safely.
Open → 2610.20809v1

Text to sql struggles with ambiguous user questions in real databases

ARCS: Towards Precise Text-to-SQL via Structured Disambiguation

Abstract: As text-to-SQL systems move beyond demonstrations toward real-world deployment, ambiguity in user questions becomes a primary source of errors. Such ambiguities are often subtle, domain- or data-specific, and can silently cause system outputs to deviate from the user's true intent. Ambiguity is traditionally addressed through conversational clarification, which is often inefficient, cognitively demanding, and poorly aligned with real-world user workflows. We propose structured disambiguation, a new paradigm in which ambiguity is resolved through explicit, constrained interactions rather than free-form dialogue. We construct ARCS (Ambiguity Resolution Corpus for SQL), the first text-to-SQL benchmark featuring naturally occurring, unconstrained ambiguities over real-world databases, with complete annotations of all valid ambiguity points, interpretations, and SQL queries. Experimental results show that text-to-SQL remains challenging in the presence of ambiguity: gpt-6-sol achieves only 51% end-to-end execution accuracy, and no open-source model exceeds 27%.

Wed 7 OctComputation and Language
The gist
When people ask computers to write database queries using natural language, their questions can be unclear or ambiguous, leading to wrong answers. The authors created a new way to resolve this ambiguity by using structured, specific interactions instead of open conversations. They made a special dataset called ARCS that contains real ambiguous questions and all possible correct interpretations. Their tests show that even the best models have a hard time dealing with ambiguity in text-to-SQL tasks.
Open → 2610.09396v1

Cost effective numerical question answering over complex tables with llms

Cost-Effective Numerical QA over Semi-Structured Table: Structuring, Resolution, Planning

Abstract: Semi-structured tables encode rich semantic information through diverse layout elements, such as hierarchical row and column headers. Answering numerical questions over such data is challenging because it requires a joint effort of accurate structural understanding of tables, question uncertainty resolution, and complex query intents interpretation. Existing methods are rarely effective in handling the above multifaceted challenges in a holistic manner; moreover, they rely on LLMs without considering financial implications. We propose SemiBonsai, a cost-effective, LLM-powered framework featuring: (1) a Multiway Layered Table Structurer that converts a table into a multiway layered tree organized by structural elements; (2) a Context-Aware Uncertainty Resolver that grounds underspecified phrases to context-coherent structural elements; (3) a Plan-Guided Reasoner that decomposes complex query intents into a query plan for reasoning; (4) a Budget-Aware Bandit Router that learns per-instance LLM utilities and optimizes LLM selection under fixed monetary constraints. Experiments on three benchmarks show that SemiBonsai improves both QA effectiveness and cost-effectiveness. Notably, the Table Structurer improves answer accuracy by 46% over alternative tree construction methods, and Router achieves a 5% average relative gain in answer accuracy compared with existing routing methods across budgets.

Tue 6 OctDatabases
The gist
Answering number questions from tables is hard when the tables have complicated layouts. The authors created a system called SemiBonsai that breaks down tables into clear parts, figures out unclear question words using context, and plans how to answer complex questions step-by-step. Their system also smartly chooses which AI to use for each question to save money. Tests show it answers questions more accurately and cheaply than older methods.
Open → 2610.07749v1

Data agents with workflows improve reliability in AI data science

Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses

Abstract: Large language model agents are increasingly deployed for data-intensive work, yet reliable data analysis requires more than general-purpose reasoning and ad hoc tool augmentation. Data Agents, equipped with workflow harnesses, offer a promising paradigm for automating the end-to-end data science lifecycle. This paper examines Data Agents from a harness-centric perspective. First, we introduce a taxonomy of Data Agents and associated data environments, organizing the literature around five functional stages: perception, planning, execution, verification, and repair. Second, we analyze the key technical routes within each stage, identifying 15 distinct approaches ranging from data structure probing to data state reconstruction. Third, we identify four open reliability problems: inactive semantic calibration, missing clarification, missing experience transfer, and the missing verification-repair repository. These problems explain why silent failures can persist even when individual components function correctly, highlighting the need for rigorous workflow harnesses and shared reliability resources. Finally, we summarize the horizontal task families of Data Agents, examine their vertical application settings, and benchmarks for evaluation, while maintaining a companion repository at https://github.com/DEEP-PolyU/Awesome-Data-Agents.

Mon 28 SeptArtificial Intelligence
The gist
Doing reliable data analysis using AI is harder than just having a smart program. The authors show that breaking down AI data work into clear steps like seeing the data, planning, doing the work, checking results, and fixing problems helps make AI data scientists more dependable. They identified common problems that cause quiet errors and suggested better ways for AI to keep track of its knowledge and fix errors. They organized what is known about data agents and shared useful tools for others to build on.
Open → 2609.35255v1

Control-based forecasting improves accuracy for changing time series

CTRL: Control-Based Time Series Forecasting with LLM-Guided Residual Learning

Abstract: Time series forecasting underpins critical decision-making across diverse domains. While large language models (LLMs) offer promising reasoning capabilities, existing LLM-based time series forecasting approaches either reduce them to numerical predictors that bypass their strengths, or allow direct forecast generation that destabilizes predictions in non-stationary settings. We introduce CTRL, a framework that decouples semantic reasoning from quantitative prediction. A frozen backbone generates base forecasts, while specialized LLM agents function as controllers that analyze backbone prediction errors through decomposed trend, seasonal, and irregular components, grounding reasoning in interpretable temporal structure. Each agent outputs compact control signals that a lightweight residual decoder translates into forecast corrections. CTRL incorporates label-free test-time adaptation that detects distribution shift from input statistics alone and readapts control signals with only 3-24 LLM calls via caching. CTRL is explicitly designed to improve robustness under non-stationary temporal dynamics and distribution shift, while remaining competitive on highly stationary time series where adaptive correction provides limited additional benefit.

Sun 20 SeptMachine LearningComputation and Language
The gist
Time series forecasting involves predicting future data points based on past observations, which is important in many areas like weather or finance. The authors show that large language models (LLMs), which are good at reasoning, have so far been used only as simple predictors or direct forecasters that struggle when conditions change. They introduce CTRL, which separates reasoning from prediction by letting a fixed forecast model make a base guess and then using LLMs as controllers to understand and adjust errors in the forecast’s trend, seasonality, and irregular parts. This method helps make predictions more stable and accurate when the data’s behavior shifts over time.
Open → 2609.23257v1

AI coders debate and agree to improve qualitative coding accuracy

How AI Coders Discuss, Disagree, and Reach Consensus: Challenges and Opportunities for LLM-Based Qualitative Coding

Abstract: The utility of AI in multi-coder qualitative coding has been widely discussed, yet little empirical evidence exists to delineate the contexts in which it performs reliably. We address this gap by quantifying the effectiveness of multi-agent LLM coding across varied qualitative datasets, revealing key contextual and structural factors that mediate coding outcomes. We developed a literature-informed baseline pipeline that enables AI agents to independently code, debate, and reconcile disagreements. Results revealed that coding accuracy depends on factors such as codebook length, qualitative data similarity, and agent disagreement. Notably, intense and unresolved debates between agents led to higher accuracy. Our analysis showed that while LLMs emulate many human discussion behaviors, they lack adaptive responsiveness to context. From these findings, we offer design recommendations for building automated coding systems. Our open-source AI discussion dataset and methodological framework lay the groundwork for advancing the design of AI-mediated automated thematic analysis.

Thu 10 SeptHuman-Computer InteractionArtificial Intelligence
The gist
Figuring out how AI can help people label text data is tricky because it depends on many things. The authors studied how multiple AI agents code, argue, and come to agreements when labeling text from different sources. They found that longer instructions, how similar the data is, and how much the AI agents disagree affect how well the AI does. Surprisingly, harder debates between AI agents sometimes meant the final answers were more accurate. The authors also noticed AI behaves a bit like humans in discussions but doesn't adapt well to changing contexts.
Open → 2609.11109v1