Business action policies balance value and constraints with causal learning
Next Best Action: Causal Constrained Offline RL for Business Action Targeting
Machine LearningArtificial IntelligenceInformation Retrieval
Summary
Choosing the best way to interact with customers, like offering discounts or promotions, is tricky because companies want to make the most money over time while following rules and being fair. The authors treat this problem like a puzzle with many rules and use a math method that learns from past data without messing with customers. Their solution picks customer actions to boost long-term value and also gives clear costs for each rule they have to follow, making it easier to understand trade-offs. They tested this with rideshare coupons and showed it could raise customer value by a good amount compared to usual methods.
What this means in practice
- •For marketing teams: Design targeted customer interventions that increase lifetime value while respecting budget and fairness constraints using offline customer data.
- •For rideshare operations teams: Improve coupon allocation policies by leveraging causal corrections in offline data to boost earnings and customer engagement.
Tested on one dataset.
Authors
Surya Ramachandiran, Yiming Li, Juan Mancilla-Caceres, Tao Ye, Jun Liu
Abstract
Next Best Action (NBA) systems must select among customer interventions (e.g., discounts, promotions, and lifecycle actions), maximize long-term economic value, satisfy complex operational constraints, and remain causally grounded. Existing approaches lack a rigorous framework for meeting these requirements jointly. We address this gap by formulating NBA as a Constrained Markov Decision Process (CMDP). We then integrate off-policy causal correction of the estimated customer dynamics directly into the Linear Programming (LP) formulation of the CMDP. The resulting policy selects actions based on customer states to maximize long-term value while explicitly enforcing business guardrails. Furthermore, the LP formulation yields shadow prices that quantify the economic trade-offs of operational constraints (e.g., fairness and user welfare), enhancing policy interpretability. We validate our proposed offline Reinforcement Learning (RL) framework on an industry case study involving rideshare coupon targeting, where customer states are defined by Recency, Frequency, and Monetary (RFM) stages. We evaluate performance using model-based off-policy evaluation, with causal dynamics estimated from a temporal holdout. Under the observed coupon budget and RFM state representation, our optimized policy improves customer lifetime value (LTV) by 19% over the estimated behavior policy and 8% over a myopic baseline. Although the demonstrated action space is binary, the proposed framework is action-agnostic, yielding a general-purpose decision module for targeting systems that manage multiple interventions.