Papers for

customer service platform developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Small models improve AI learning with selective large model help

SCALE: Small-Model-First Concept Abstraction with LLM-Guided Experience Internalization

Abstract: Autonomous agents must continually acquire reusable knowledge from experience without relying on costly large language model (LLM) reasoning for every decision. However, existing experience-based approaches often struggle to transform accumulated observations and reasoning into compact knowledge that lightweight models can independently apply. This paper proposes SCALE, a small-model-first framework that integrates selective LLM reasoning, experience-driven concept abstraction, and knowledge internalization. A lightweight model handles routine predictions, while prediction errors and accumulated cross-episode evidence selectively trigger LLM reflection. The LLM analyzes relevant historical experiences, generates positive and negative counterfactual examples, and proposes reusable concept hypotheses. Independently validated concepts are progressively incorporated into hierarchical neural representations, enabling lightweight models to exploit increasingly abstract knowledge. Meanwhile, mastered experiences are excluded from active memory while their original records remain available in a complete archive. Experiments across ten random seeds show that historical recall improves held-out accuracy from 87.11% to 98.28%, hierarchical concept knowledge increases difficult-case accuracy from 55.33% to 66.47%, and abstract archiving reduces active experience records from 6,000 to 1,528.

Mon 9 NovArtificial IntelligenceRobotics
The gist
AI programs often rely on large language models (LLMs) that are powerful but slow and expensive for everyday tasks. This paper presents SCALE, a method that lets smaller, faster models handle simple decisions while using the larger models only when mistakes happen or tricky situations arise. The large models help by looking back at past experiences and suggesting new ideas, which are then tested and taught to the smaller models. This approach makes AI systems more efficient by storing important knowledge in a compact form and reducing unnecessary thinking.
Open → 2611.12120v1

Sales agent trained on real chats competes well with humans and AI

Selling in the Wild: Benchmark and Randomized Field Evidence for a Sales Agent Trained on Commercial Conversations

Abstract: We test whether Javo, a sales agent fine-tuned on 4.6 million consented, de-identified commercial conversations, sells better than a frontier general-purpose model and better than human sales associates. The evaluation has four parts. Study 1 scores the agent on two public sales benchmarks, SalesLLM and SmartSalesBench, against 10 frontier and open-weight models. Study 2 is a randomized controlled trial on 100 live websites in 14 industries (e-commerce, travel and hospitality, and high-consideration services), run from 1 May to 30 September 2026, in which 14.8 million visitors were randomized to no agent, Javo, or a frontier-model agent with an identical interface, catalog tools and guardrails. Study 3 isolates the effect of the model by comparing those two agent arms. Study 4 compares Javo with each site's own human associates on 20 sites with staffed live chat. Relative to no agent, Javo changed revenue per visitor by +10.0% (95% CI +5.1 to +14.9). Among engaged visitors, it changed conversion by +21.6% relative to the frontier-model agent (95% CI +16.9 to +26.3). Against human associates, its engaged conversion was 11.26% versus 11.20% (non-inferior, margin -7%, one-sided p = 0.004; no significant superiority), at 1.8% of the cost per conversation. Return rates, unsupported product claims and complaint rates met the prespecified safety guardrails.

Sun 18 OctInformation RetrievalComputers and Society
The gist
Selling products online often involves chatting with customers, either by humans or automated agents. The authors tested Javo, an AI sales agent trained on millions of real sales conversations, across public benchmarks and live websites. Javo increased sales revenue by 10% compared to having no agent and outperformed a general AI model by over 20% in converting visitors to buyers. When compared to human sales associates, Javo performed similarly while costing far less per conversation. It also maintained safe customer service standards.
Open → 2610.25461v1

Improving bilingual banking search accuracy with evidence gating

Evidence-Gated Bilingual RAG for Reliable Consumer Banking Search

Abstract: Consumers often describe what they need rather than use the terminology found on a banking website. For example, a natural-language request ("a card for travel") may not share the keywords of the relevant product page, and the problem is harder in bilingual services, where the same intent may be expressed in Arabic or English. This paper presents an evidence-gated bilingual retrieval-augmented generation (RAG) framework for consumer banking search that operates entirely on local infrastructure. The framework combines structured web-content refinement, hybrid dense-lexical retrieval, intent-aware evidence selection, cross-encoder reranking, and a three-outcome Answer / Related Mismatch / No Answer decision mechanism. On 210 URL-labeled Arabic and English queries, the proposed approach improves Top-1 hit rate from 50.48% to 79.05% and MRR@5 from 0.571 to 0.797 over a hybrid baseline. A separate 26-case end-to-end evaluation achieved 69.2% strict success, with 72.7% of generated answers classified as supported by an automated faithfulness judge. These results are preliminary: the retrieval queries are paraphrases of a small reference set, and the end-to-end test is small and uses an automated rather than human judge, so they do not establish human-verified answer correctness or generalizable performance.

Thu 15 OctComputation and Language
The gist
People searching banking websites often use everyday language that doesn’t match official website terms, making it hard to find the right info. This problem is even harder when users use two languages like Arabic and English. The authors developed a new system that carefully picks and checks information from banking websites to better understand and answer user requests in both languages. Their tests show big improvements in matching searches to the right banking pages and in generating trustworthy answers, though more testing is needed to confirm results fully.
Open → 2610.22652v1

Speech encoder learns multiple voice traits at once with multi-teacher method

Learning Speech Attribute Representations via Multi-Teacher Distillation

Abstract: Recent post-training approaches have shown that self-supervised speech encoders can be adapted to learn a single utterance-level speech attribute through knowledge distillation. In this work, we investigate whether a single speech encoder can simultaneously preserve multiple complementary utterance-level speech representations. We propose a multi-attribute multi-teacher distillation framework that learns semantic, speaker, language, and emotion representations from four frozen teacher models using a shared SSL backbone and attribute-specific branches. Experiments on multilingual semantic retrieval, speaker verification, language identification, and speech emotion recognition show that the proposed model achieves performance close to dedicated single-task students while requiring only a single shared encoder. Furthermore, the learned layer aggregation strategies reveal that different speech attributes naturally emerge at different depths of the SSL encoder, providing new insights into the organization of speech representations.

Thu 15 OctComputation and Language
The gist
Speech systems usually learn one thing at a time, like understanding words or recognizing the speaker. This paper explores whether one system can learn many voice features at once, such as what is said, who is speaking, the language, and the emotion. The authors use a method where a main system learns from four experts specializing in each trait simultaneously. The result is a single speech encoder that performs nearly as well as systems trained only for one task, but much more efficiently. They also found that different voice traits are best recognized at different levels inside the system, offering clues about how speech information is organized.
Open → 2610.21941v1

Framework measures response variability and risk in large language models

A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models

Abstract: Large language models are stochastic by nature. Repeated executions under the same experimental conditions may produce different responses. We propose a framework that can be applied across the model lifecycle, from task design to residual behavior and continuous monitoring. The method consists of building an empirical response distribution estimated through Monte Carlo simulation. Semantic interventions are introduced to study how these distributions change. We illustrate the framework with two different tasks, borrower classification and music album recommendation, and quantify the resulting distributional changes. The framework is linked to model risk management and governance by measuring robustness and control effectiveness, and characterizing residual behavior. Since it relies only on observed responses, it can also be applied to black-box models accessed through commercial APIs.

Thu 8 OctMachine Learning
The gist
Large language models can give different answers each time you ask the same question because they work in a random way. The authors propose a way to study all possible answers by running the model many times and seeing how the answers change. They also test what happens when they try to steer or change the model’s behavior in specific ways. This helps people understand how reliable the model is, how well it can be controlled, and what unexpected behaviors might remain. Their approach works even if you only have access to the model’s outputs, like when using online services.
Open → 2610.12718v1

Gated memory improves fact storage quality in conversational AI systems

Gated Memory: Admission-Controlled Memory Formation for Conversational AI

Abstract: Personalized conversational AI relies on long-term memory systems that extract facts from user utterances and store them in persistent vector stores. Despite progress in retrieval, deduplication, and lifecycle management, the formation stage, the moment a fact is first written to storage has received almost no principled attention. We identify this as the binding constraint on memory quality in production systems. Critical contextual signals, such as the distinction between a permanent user attribute and a transient situation, exist only in the original utterance and are irreversibly lost the moment extraction produces a subject-relation-object triple. No downstream process can recover them. We propose Gated Memory, a lightweight, modular formation framework that interposes two decision checkpoints between conversation and storage: an admission gate that evaluates every candidate fact against the full utterance context before extraction runs, and a conditional enrichment stage that grounds admitted facts through an entity scope taxonomy with privacy constraints. The gate evaluates only the current exchange while using prior turns as read-only reference context, and produces a structured formation record. Admitted content is decomposed into atomic facts, each categorized, tagged with provenance (directly stated versus inferred), scoped to its condition of applicability, and grounded in resolved time and place, subject to a constraint that no entity absent from the context may be asserted. On the LoCoMo-10 benchmark with atypical emotional density in utterance data, Gated Memory achieves an overall +2.6% relative improvement in LLM-judge accuracy over a strong baseline with identical retrieval and generation, establishing formation quality as a measurable constraint on memory performance.

Thu 8 OctComputation and LanguageArtificial IntelligenceInformation Retrieval
The gist
Conversational AI systems need to remember important facts about a user over time, but deciding which facts to save can be tricky. The authors found that current methods lose important context from the original conversation when storing facts. They propose a new system called Gated Memory that carefully decides which facts to keep and adds extra information like when and where the fact applies. This method improved memory accuracy in tests with emotionally rich conversations.
Open → 2610.11270v1

SkillFormer improves audio models by splitting and routing distinct skills

SkillFormer: Skill-Decomposed Adaptation for Audio Language Models

Abstract: Audio language models must handle dozens of distinct skills, from pitch comparison and speaker counting to musical tempo estimation and emotion recognition. Joint training on all skills at once causes interference: gains on one skill often come at the cost of another. We propose \textbf{SkillFormer}, which decomposes audio understanding into skill-specific low-rank adapters and composes them at inference time through a learned router. The router examines the question to decide which adapters to activate and how much weight each should carry, so that a pitch query engages different parameters than a genre classification query. An alternating training schedule updates each adapter on its own skill cluster before jointly calibrating the router, preventing the gradient conflicts that arise in standard multi-task optimization. SkillFormer adds fewer than 4\% of the base model's parameters and requires no changes to the audio encoder or language backbone. Evaluated on three architecturally distinct models across MMSU, MMAU-Pro, and MMAR, it raises the average accuracy by 2.5 to 4.1 points, with balanced gains across perception, reasoning, and semantic subcategories.

Tue 6 OctSoundMachine Learning
The gist
Audio language models need to perform many different tasks, like recognizing emotions or counting speakers, but training them all together can cause problems where improving one task hurts another. The authors propose SkillFormer, which breaks down audio understanding into separate skill-specific components that are combined dynamically based on the question being asked, so each task uses its own tailored parameters. This design prevents conflicts during training and improves overall accuracy without major changes to existing models. They tested SkillFormer on different models and datasets, showing consistent improvements across a range of audio understanding abilities.
Open → 2610.07533v1

ForkPilot improves long-task AI decisions cutting token use nearly in half

ForkPilot: Self-Evolving Policy for Retrospective Search in Long-Horizon Agents

Abstract: Interactive language-model agents increasingly solve complex tasks through long-horizon, multi-call reasoning, where errors in beliefs or actions can compound across tool interactions. Retrospective search can recover from such failures but is prone to misallocation. Delayed outcomes obscure the contribution of intermediate search decisions, leading to Attribution Complexity, while evolving execution evidence leads to Adaptation Complexity, where previously learned estimates become stale. To address these challenges, we first introduce Search Value Dynamics (SVD), which characterizes the evolving trade-off between the gain and cost of retrospective search. Building on SVD, we propose ForkPilot, a self-evolving two-stage policy-learning framework. In the first stage, ForkPilot learns a search-value policy offline from completed trajectories through automatically constructed outcome comparisons. In the second stage, it makes search decisions based on current observations and then self-evolves by incorporating newly completed trajectories into subsequent policy updates. We evaluate ForkPilot across 6 diverse benchmarks and 7 widely used LLM backbone families, including four open-source families, GPT-5.6 Sol, and Opus 4.8 in a production agentic system, against 9 competitive baselines, including a real-world harness deployment used by hundreds of thousands of paid users. ForkPilot achieves comparable state-of-the-art performance while reducing token usage by up to 59.2%, demonstrating its efficacy.

Sun 4 OctArtificial Intelligence
The gist
Sometimes AI agents make mistakes when doing tasks that take many steps, because early errors build up later. The authors found that looking back at past decisions to fix mistakes is tricky because it’s hard to tell which earlier steps caused problems, and past experience can become outdated. They created ForkPilot, a new method that learns when to look back and fix earlier steps by training itself on past attempts and improving over time. In tests across many different AI systems and tasks, ForkPilot performed as well as top methods but used much less computing effort.
Open → 2610.04889v1

Reliable multi-turn business agent evaluation with modular LLM judges

Designing Reliable LLM-as-a-Judge Measurement Systems for Multi-Turn Business Agents

Abstract: Many LLM-as-a-judge evaluations score fixed outputs under a fixed task definition. Production multi-turn business agents instead require a maintained measurement system: correctness depends on business-specific facts and procedures, outcomes emerge across turns, and failures must be attributed to either agent capability or missing business knowledge before they are actionable. We present an integrated methodology spanning evaluation specification, modular LLM judges, intent-preserving user simulation, and human-in-the-loop governance. The specification defines conversation-level end states and actionable failure ownership. Atomic judges share versioned evidence and feed an explicit aggregation graph. The simulator is released only after task-preservation and stability checks. Independent human audits estimate measurement fidelity, renew tiered reference sets, and route disagreements to label correction, guideline revision, or judge improvement. Production studies show that system-level fidelity improved across repeated audits, that human reviewers and automated judges improved together under the shared feedback loop, and that their combined workflow had the strongest descriptive performance in both reported task-completion settings. Because the studies are observational and the human reference itself required revision, these findings demonstrate operational usefulness rather than causal or universal superiority. The contribution is a practical framework for making multi-turn agent measurement reliable, actionable, and maintainable as the evaluated system and its evidence evolve.

Sun 27 SeptArtificial Intelligence
The gist
Evaluating AI business agents over multiple conversation turns is tricky because success depends on complex facts and steps that unfold over time. The authors developed a method that uses small, specialized AI judges combined with simulations and human checks to measure how well these agents perform. Their system helps figure out if problems come from the AI’s abilities or missing business knowledge, making fixes easier. Tests in real scenarios showed this method improves judging accuracy and helps humans and AI judges learn from each other.
Open → 2609.33955v1

Generative AI agents need resilience and care over time

Finishing the Task Is Not Enough: Evaluating Agent Resilience and Considerate Participation under Accumulating Challenge

Abstract: Sustained deployment of generative AI agents requires more than isolated task success. Agents must remain useful across repeated interactions, changing conditions, and dependencies on people within shared workflows, especially as technical, human, and operational disruptions accumulate over time. We propose operational resilience and considerate participation as two complementary aspects of evaluating such agents: the former captures how agents recover from blocked work while preserving progress and communicating their limits, and the latter captures how their adaptation accounts for affected people, role boundaries, and the surrounding workflow. Yet both remain underexplored under accumulating challenge. We study 120 simulated healthcare trajectories across two generative AI models and twelve stakeholder-derived tasks under light, medium, and heavy challenge. We compare textual action plans, prompted internal assessments, and quantitative structured workload and affect reports to examine how agent behavior and reported state change as challenge accumulates. Regarding operational resilience, agents shift from self-directed recovery toward greater human dependence, while reporting increasing workload and negative affect in structured reports but seldom expressing strain in textual responses. Regarding considerate participation, agents broaden from task-focused adaptation toward task reframing, attention to others, role-boundary adjustment, and wider coordination, with distinct patterns across actions and internal assessments. From these findings, we derive five deployment dilemmas involving persistence, attention, role boundaries, state disclosure, and escalation that require stakeholder specification, further informing technical implications for learning, situated evaluation, and embodied adaptation.

Wed 9 SeptArtificial IntelligenceHuman-Computer InteractionMultiagent Systems
The gist
Getting AI to finish a task isn’t enough for long-term use, especially in places like healthcare. The authors found that AI agents need to bounce back from problems and work well with people who depend on them. They studied how AI handles challenges over many interactions and saw that agents change how they recover and how they pay attention to others. The paper highlights important trade-offs when deploying AI to help people in ongoing, real-world workflows.
Open → 2609.10724v1