Credit assignment method improves multi-action retrieval in sponsored search
CASI: Credit Assignment via Set Intervention for Agentic Retrieval
Artificial IntelligenceInformation RetrievalMachine Learning
Summary
When computer systems make several decisions together, it can be hard to tell how much each choice contributed to the overall result. The authors propose CASI, a way to break down credit fairly among multiple decisions by testing alternative groups of actions under the same conditions. They use a technique called Shapley values to assign credit and incorporate it into learning better decision policies. On sponsored search data, CASI leads to improved policies and even showed a positive impact when tested live.
What this means in practice
- •For online advertising teams: Optimize bidding strategies by fairly assigning credit to sets of ad placements for better sponsored search outcomes.$Commercial implications: Enables building ad auction optimization products that improve advertiser value by accurately distributing reward among multiple retrieved actions.
- •For recommendation system engineers: Improve retrieval policies in systems that generate multiple outputs by accurately attributing credit to subsets of retrieved items.
Authors
Xinyi Cheng, Tianfu Zhang, Jiwen Li, Xuesen Zhang, Shihao Yu, Pengfei Zhu, Renhuan, Heda Wang
Abstract
Large language models increasingly generate multiple retrieval actions jointly, while downstream systems often provide only one reward for the resulting set. Broadcasting this reward provides a valid score-function gradient estimator for the expected set-level objective, but does not determine a coalition-dependent within-set attribution. We introduce CASI (Credit Assignment via Set Intervention for Agentic Retrieval), which constructs the missing counterfactual information through snapshot-grounded set intervention: generate and retrieve once, hold replayable snapshot components fixed, and evaluate alternative subsets under common conditions. Shapley aggregation converts these marginals into within-set credit, which bounded zero-sum advantage shaping incorporates into policy optimization. For m >= 4, singleton, leave-one-out, and full-set endpoints do not generally determine the Shapley allocation; under additive set values, extra coalition evaluations are redundant. In a five-action sponsored-search setting, exhaustive auction replay over 1,200 requests reveals substantial coalition dependence. Under the training-time evaluator, CASI achieves the highest median cosine similarity to a replay-defined Shapley reference among evaluated credit rules; under a sensitivity backend, CASI does not outperform Singleton. Across five independent training runs, CASI-trained policies achieve higher held-out auction-replay set value than Singleton- or Broadcast-trained policies. In a randomized seven-day production A/B test on 10% of traffic, the CASI-trained policy yields an observed 3.51% lift in the primary advertiser-value metric over production control.