Right-Sizing LLM-Agent Decomposition in VAT Determination: A Pilot Controlled Sweep
2026-08-24 • Multiagent Systems
Multiagent SystemsArtificial IntelligenceSoftware Engineering
AI summaryⓘ
The authors studied how to best split a complex task of determining VAT rules across different agents, comparing one strong agent to several smaller, specialized ones. They kept everything else the same but changed how the work was divided and ran thousands of tests to measure accuracy and reliability. Their results showed mixed outcomes: two middle-ground approaches were more accurate but didn't clearly beat the single-agent setup in all tests. They also found that when errors were introduced, wider scoped restarts helped recover results better than fragmented approaches. Overall, the study offers a careful pilot method for deciding how to split tasks among agents, sharing all their data and tools.
LLM agentsVAT determinationreverse chargetask decompositionorchestratorprompt budgetfailure injectiondeterministic rule enginepilot studyagent configuration
Authors
Pedro Santos
Abstract
Recent LLM-agent systems make conflicting design bets: decompose work across many narrow agents, or use one strong tool-using agent. This pilot studies that choice on bounded cross-border VAT determination with reverse charge, where every case has an oracle label and each intermediate decision is independently scoreable. We hold the activity surface fixed (subtasks, tools, I/O schemas, validation checks, orchestrator, base model, and merge policy) and vary only the assignment of subtasks to workers across four orchestrated configurations, from one wide worker to five narrow ones, against S0, a tuned no-orchestrator single agent, with a deterministic rule engine as oracle. The program spans 4,400 runs: a 40-case, five-repeat main sweep, matched-token arms separating prompt-budget from agent-count effects, and three failure-injection arms, all judged against pre-registered falsification criteria. The two intermediate configurations lead on accuracy (0.830, against endpoints at 0.720 and 0.770) but miss the pre-stated bar against the fine endpoint, so the intermediate-optimum hypothesis remains unsupported at pilot scale. The single agent does not Pareto-dominate the orchestrated set. The matched-token criterion fires: the budget-matched single agent lands 6.5 points below the leader, but the interval includes zero, so any advantage is consistent with a prompt-budget explanation. Under injection, availability faults are absorbed at every granularity, with wide-scope restart over-recovering its baseline by +0.160, while one schema-conforming hallucinated record degrades every configuration and inverts the ordering, hitting fragmented configurations hardest. The contribution is a bounded, preregistered pilot heuristic for right-sizing decomposition (place one partition boundary at the dependency-layer midpoint), released with oracle, dataset, harness, raw traces, and analysis pipeline.