ComboShoppingBench: Evaluating LLM Agents for Budget-Constrained Basket Shopping with Coupons
2026-08-10 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors created ComboShoppingBench, a test for AI systems to build shopping baskets with multiple items that fit certain rules like budgets and coupons. This is different from just picking one product because the items must go well together and follow store rules. They designed a way to check both if the basket makes sense and if it follows all the constraints, using a mix of AI judges and rule-based checks. Their tests show that even advanced AI models find this task difficult, indicating room to improve shopping assistants that handle complex orders.
basket constructioncomplementary itemsconstraintscoupon optimizationbudget complianceLLM evaluationsemantic satisfactionshopping agentsimulated commercevalidation
Authors
Adrian Li, Kelong Mao, Yudong Guo, Heming Xia, Xinwei Yang, Lirui Luo, Jace Wong, Pu Yao, Sulong Xu, Simiu Gu
Abstract
Real-world shopping often requires constructing a basket of complementary items rather than retrieving a single product. Such combo-shopping tasks arise in device setup, meal preparation, event planning, and group takeout ordering, requiring joint reasoning about item compatibility, availability, store-level requirements, delivery fees, coupons, and budgets. Evaluation is challenging because multiple baskets may satisfy the same request, making exact-match metrics unsuitable, whereas semantic evaluation alone cannot detect infeasible orders, invalid coupon combinations, or incorrect payments. We introduce ComboShoppingBench, an agentic shopping benchmark for open-ended yet verifiable basket construction in a simulated commerce and takeout environment. During task synthesis, an exploration agent constructs a feasible and semantically coherent basket of purchasable products; this witness guides the generation of coupons, budget constraints, user queries, and aligned evaluation rubrics. During evaluation, LLM judges assess semantic satisfaction, response quality, and claim faithfulness, while deterministic validation checks product-ID validity, budget compliance, and coupon optimality. Experiments with diverse LLM agents demonstrate that even strong agents struggle on ComboShoppingBench, highlighting substantial room for improvement in reliable, constraint-aware combo shopping.