Churn prescription improves with counterfactual reasoning and retrieval
CARRE: Counterfactual Action Retrieval and Reason Evaluation for Explainable Churn Prescription
Computation and Language
Summary
Customers at risk of leaving a company are often identified, but companies do not always know which actions will keep them and why. The authors present CARRE, a way to suggest specific actions to retain customers by combining examples of retention strategies with predictions on how those actions might reduce churn risk. It also explains why an action is chosen in a clear way. CARRE performs better than some existing explanation methods on a telecom customer dataset, offering more useful and cost-effective recommendations.
What this means in practice
- •For customer retention teams: Suggest specific, cost-effective actions to reduce customer churn risk and explain their rationale in customer support contexts.
- •For marketing analysts: Generate understandable explanations linking customer data changes to churn risk for targeted marketing interventions.
Tested on one dataset.
Authors
MinJoo Kim, SanJin Park, SeungHwan Cho
Abstract
Churn models typically identify high-risk customers but do not specify which feasible retention action should be considered or why that action is appropriate. We present CARRE (Counterfactual Action Retrieval and Reason Evaluation), a three-stage framework that combines retrieval-augmented candidate generation, cost-aware counterfactual scoring, and large language model (LLM) reasoning. CARRE retrieves a predefined catalog of retention actions, estimates model-predicted churn-risk changes under explicit feature transformations, and generates a structured churn reason and a profile-grounded explanation for the selected action. On the IBM Telco Customer Churn dataset, CARRE achieves 79.8% greater mean model-predicted risk reduction than the plain SHAP baseline and 80.4% greater reduction than the cost-controlled SHAP+Cost baseline across 313 high-risk test cases; its cost-normalized efficiency is 10.5% higher than that of plain SHAP. On a 136-case reason-stratified evaluation sample, diagnosis-driven prompt refinement increases weak-label agreement from 79.4% to 90.4%, with no auxiliary-plan constraint violations; because the same sample was used for error diagnosis and re-evaluation, the post-refinement result is not an independent estimate of generalization. For 135 explanations generated using the pre-refinement v2 reason outputs, two cross-vendor LLM judges assign mean scores ranging from 4.02 to 5.00 out of 5, although one judge saturates on actionability, and a deterministic audit finds no contradictions among 66 verifiable profile claims. Retrieval ablations show that k=5 provides the best evaluated compromise between high candidate coverage and downstream reasoning agreement in this dataset. These results illustrate how retrieval, model-based counterfactual scoring, and language generation can be separated and jointly evaluated in a prototype churn-prescription pipeline.