ReTrace cuts costs and speeds up business automation workflows
ReTrace: Distilling Tool-Using Agents into Planner-Free Workflows
Artificial IntelligenceMachine LearningMultiagent Systems
Summary
Running the same business procedures over and over is slow and expensive when relying on AI every step of the way. The authors introduce ReTrace, a system that learns from a few AI-guided runs to create a rule-based workflow that doesn’t need AI for each step. This saves time and money while staying almost as effective as the AI-driven method. ReTrace can adapt its rules for different software platforms and repair itself when errors happen.
What this means in practice
- •For enterprise automation teams: Use ReTrace to convert AI-driven business procedures into faster, cheaper rule-based workflows without losing much accuracy.
- •For software developers integrating ai tools: Compile human-readable rules from AI agent logs for different automation platforms to reduce dependency on AI inference for every workflow step.
Authors
Kaiyuan Hou, Yang Ping Kuo, Indranil Bhattacharya, Rohith Nama, Arushi Gupta, Carlos D. Castillo, Prathap Ramachandra, Xiaofan Jiang, Francesco Carbone
Abstract
Business process automation runs the same Standard Operating Procedure (SOP) again and again on new inputs. Large Language Model (LLM) agents can follow an SOP written in natural language, but they call an LLM to choose every step of every run, which adds latency and cost at scale. We present ReTrace, which turns an SOP and a few logged agent runs into a reusable workflow in which code, not an LLM, selects the next step. The agent's reasoning, the explanation it writes with each tool call, often states why it took an action; the tool calls alone do not. ReTrace extracts the condition behind each step, merges these conditions across runs into a compact rule document, and generates an executable workflow from it. Only this last stage depends on the platform, so rule documents can be compiled for different orchestration platforms. On $79$ procedures from our production platform, ReTrace workflows reach $93.8\%$ completion, against $96.8\%$ for the agent, at $24\%$ lower cost per task. On $τ^2$-Bench, they reach $75.8\%$ task success (agent: $80.6\%$) and run in $1.3$s instead of $170$s; three rounds of repair on training failures lift $k{=}2$ workflows from $66.7\%$ to $81.2\%$. On eight SOP-Bench procedures, ReTrace without repair trails few-shot generation ($80.6\%$ vs. $83.6\%$), but repair from labeled training failures lifts it to $87.8\%$ at $0.40$s per task, compared to $85.5\%$ in $46.6$s for the agent.