Papers for

business solution developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Benchmark shows agents struggle with real business task delivery

FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests

Abstract: Real-world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience. Forward-deployed engineers (FDEs) must elicit these requirements through dialogue and iteratively optimize solutions until they meet customer acceptance criteria. Existing optimization benchmarks typically start from explicit objectives, while clarification benchmarks often introduce information gaps into existing tasks. We introduce FDE-Bench, the first benchmark built from real customer projects to evaluate agents on end-to-end delivery tasks that require interactive requirement clarification and iterative optimization. Its 49 cases span combinatorial optimization and machine learning. Clarification targets are distilled from discussions between frontline practitioners and FDEs. Agents elicit missing requirements from a simulated customer, and case-specific evaluators assess final artifacts against acceptance criteria using customer-accepted FDE solutions as quality references. Across 18 agent configurations, the highest mean delivery score is only 0.5510, compared with an FDE reference of 1.0. The highest delivery success rate (DSR), defined as the fraction of planned runs reaching this reference, is only 6.12%. These results show that reliable FDE-level end-to-end delivery remains out of reach for the evaluated agents on underspecified real-world business tasks. The benchmark project page is available at https://baidubce.github.io/FM-Agent/FDE-Bench/.

Sat 10 OctArtificial IntelligenceMultiagent Systems
The gist
Many business requests don’t clearly explain what is needed because workers rely on experience they can’t fully describe. The authors created a benchmark called FDE-Bench using real cases where engineers have to talk to customers to figure out missing details and improve solutions step-by-step. They tested different AI agents on these tasks, but none performed as well as human engineers. This means current AI still finds it hard to fully handle complex business tasks where requirements are unclear.
Open → 2610.14539v1