Benchmark evaluates ai sales agents against tough buyer behaviors
SmartSalesBench: A Resistance-Aware Multi-Turn Benchmark for Evaluating LLM Sales Agents
Artificial IntelligenceComputation and Language
Summary
Many tests for AI sales helpers assume customers are easy to work with, but real buyers often push back. The authors created SmartSalesBench, a special test that puts AI sales agents in long conversations with buyers who resist and challenge them. It includes different types of AI agents, buyer personalities, and products to try. The authors also designed ways to measure how well the AI handled tough conversations and used both automatic scoring and human checks to make sure results are reliable.
What this means in practice
- •For llm developers: Test and improve AI sales agents' ability to handle challenging, resistant buyers across realistic multi-turn conversations.
- •For customer service platform teams: Evaluate and select conversational agents that can persist through buyer resistance and increase realistic sales interactions.
Authors
P N Bhargav Teja, Velpula Lokesh Reddy, Madhu Basumata, Susen Sarkar, Rajasekhara Reddy Duvvuru Muni
Abstract
We present SmartSalesBench, the first multi-turn conversational benchmark explicitly designed to evaluate large language model (LLM) sales agents against psychologically grounded, resistant buyer personas. Existing task-oriented dialogue benchmarks assume cooperative users and measure task completion; SmartSalesBench inverts this assumption, placing agents against personas programmed to resist and challenge across conversations of up to twenty turns. The benchmark includes five agent profiles, twelve buyer personas, fifteen smartphone products across six price tiers, and four controlled experiments with 3 iterations totaling 13,680 sessions. Both sales-agent and buyer persona generations are grounded through retrieval-augmented generation (RAG). We introduce nine evaluation metrics grouped into three tiers: primary outcome metrics (CR, BRS, RIR), secondary quality metrics (SMS, PFS, ETI), and exploratory metrics planned for future reporting (ITS, ARI, SPI). Our evaluation combines automated LLM-as-judge scoring with internal human assessment (n = 50, \k{appa} = 0.71, Pearson r = 0.81 with BRS).