Papers for

software engineers in ai automation

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

E-commerce AI agents struggle to recover from silent tool errors

RITE: Evaluating End-to-End Resilience to Silent Tool Errors in E-Commerce Agentic Systems

Abstract: Tool-using LLM agents increasingly answer e-commerce queries through multi-step tool interactions. The expected output of a tool call can shift between agent development and execution, allowing a plausible but incorrect value to be returned without an explicit failure signal. These silent errors may propagate through subsequent decisions into final answers. To study agents' resilience across this end-to-end process, we formulate Silent-error Resilience Task, an interactive task that evaluates whether they complete the original query after a silent error corrupts an intermediate state. We introduce RITE, a benchmark built from human-authored industrial workflows and production-log-derived inputs. To enable controlled and reproducible evaluation, we use a semantic error taxonomy to inject errors into fixed workflow prefixes, freeze recorded tool behavior, and let agents continue freely. It contains 167 matched units across nine e-commerce workflows, yielding 501 silent-error, correct-value, and explicit-failure instances. Across 10 recent LLMs, silent errors roughly halve task success and are consistently harder than explicit failures. Recognition, use of an available verification path, and integration of the resulting evidence form successive bottlenecks. We also propose a light State Checker as an intervention to test whether resilience can be improved.

Tue 20 OctArtificial Intelligence
The gist
In e-commerce AI systems that use multiple tools step-by-step, mistakes can quietly slip in when a tool returns a believable but wrong answer without signaling a failure. These hidden errors can affect later decisions and final results. The authors created a test called RITE to check how well AI agents handle these silent mistakes and whether they can still complete tasks correctly. They found that these silent errors significantly reduce success rates and are harder to detect compared to clear failures. They also tested a simple checking tool to help improve error recovery.
Open → 2610.27777v1