Staying Power of Churn Prediction Models

In this paper, we study the staying power of various churn prediction models. Staying power is defined as the predictive performance of a model in a number of periods after the estimation period. We examine two methods, logit models and classification trees, both with and without applying a bagging procedure. Bagging consists of averaging the results of multiple models that have each been estimated on a bootstrap sample from the original sample. We test the models using customer data of two firms from different industries, namely the internet service provider and insurance markets. The results show that the classification tree in combination with a bagging procedure outperforms the other three methods. It is shown that the ability to identify high risk customers of this model is similar for the in-period and one-period-ahead forecasts. However, for all methods the staying power is rather low, as the predictive performance deteriorates considerably within a few periods after the estimation period. This is due to the fact that both the parameter estimates change over time and the fact that the variables that are significant differ between periods. Our findings indicate that churn models should be adapted regularly. We provide a framework for database analysts to reconsider their methods used for churn modeling and to assess for how long they can use an estimated model.

[1]  P. Verhoef,et al.  CRM in Data-Rich Multichannel Retailing Environments: A Review and Future Research Directions , 2010 .

[2]  P. Verhoef,et al.  Marketing Communication Drivers of Adoption Timing of a New E-Service among Existing Customers , 2007 .

[3]  Robert C. Blattberg,et al.  Can We Predict Customer Lifetime Value? Can We Predict Customer Lifetime Value , 2004 .

[4]  Edward C. Malthouse,et al.  Ridge regression and direct marketing scoring models , 1999 .

[5]  Sunil Gupta,et al.  Valuing customers , 2007 .

[6]  Robert C. Blattberg,et al.  Customer Lifetime Value: Empirical Generalizations and Some Conceptual Questions , 2009 .

[7]  Nissan Levin,et al.  Applying neural computing to target marketing , 1997 .

[8]  Dirk Van den Poel,et al.  Predicting customer retention and profitability by using random forests and regression forests techniques , 2005, Expert Syst. Appl..

[9]  Tuck Siong Chung,et al.  Marketing Models of Service and Relationships , 2006 .

[10]  Dominique Haughton,et al.  Direct marketing modeling with CART and CHAID , 1997 .

[11]  Dominique M. Hanssens,et al.  Creating lift versus building the base: current trends in marketing dynamics , 2009 .

[12]  Rajkumar Venkatesan,et al.  A Customer Lifetime Value Framework for Customer Selection and Resource Allocation Strategy , 2004 .

[13]  Katherine N. Lemon,et al.  The Customer Pyramid: Creating and Serving Profitable Customers , 2001 .

[14]  Katherine N. Lemon,et al.  The Theoretical Underpinnings of Customer Asset Management , 2002 .

[15]  Ruth N. Bolton,et al.  A Dynamic Model of the Duration of the Customer's Relationship with a Continuous Service Provider: The Role of Satisfaction , 1994 .

[16]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  Philip Hans Franses,et al.  Quantitative Models in Marketing Research , 2001 .

[18]  Kristof Coussement,et al.  Faculteit Economie En Bedrijfskunde Hoveniersberg 24 B-9000 Gent Churn Prediction in Subscription Services: an Application of Support Vector Machines While Comparing Two Parameter-selection Techniques Churn Prediction in Subscription Services: an Application of Support Vector Machines While Comparin , 2022 .

[19]  A. Athanassopoulos.  Customer Satisfaction Cues To Support Market Segmentation and Explain Switching Behavior , 2000 .

[20]  Christophe Croux,et al.  Bagging and Boosting Classification Trees to Predict Churn , 2006 .

[21]  Sungzoon Cho,et al.  Response models based on bagging neural networks , 2005 .

[22]  Robert C. Blattberg,et al.  Database Marketing: Analyzing and Managing Customers , 2008 .

[23]  Dominique Haughton,et al.  Application of multiple adaptive regression splines (MARS) in direct response modeling , 2002 .

[24]  Edward C. Malthouse,et al.  Improving predictive scoring models through model aggregation , 2008 .

[25]  Dirk Van den Poel,et al.  Customer base analysis: partial defection of behaviourally loyal clients in a non-contractual FMCG retail setting , 2005, Eur. J. Oper. Res..

[26]  Peter C. Verhoef,et al.  The commercial use of segmentation and predictive modeling techniques for database marketing in the Netherlands , 2003, Decis. Support Syst..

[27]  Peter S. Fader,et al.  How to project customer retention , 2007 .

[28]  L. Breiman.  Heuristics of instability and stabilization in model selection , 1996 .

[29]  Wei-Yin Loh,et al.  A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms , 2000, Machine Learning.

[30]  Peter C. Verhoef,et al.  Modeling CLV: A test of competing models in the insurance industry , 2007 .

[31]  Akhil Kumar,et al.  An empirical comparison of neural network and logistic regression models , 1995 .

[32]  D. Collings,et al.  Valuing customers , 2005 .

[33]  P. Verhoef.  Understanding the Effect of Customer Relationship Management Efforts on Customer Retention and Customer Share Development , 2003 .

[34]  P. K. Kannan,et al.  Implications of loyalty program membership and service experiences for customer retention and value , 2000 .

[35]  Jeffrey S. Simonoff,et al.  Tree Induction Vs Logistic Regression: A Learning Curve Analysis , 2001, J. Mach. Learn. Res..

[36]  Philip Hans Franses,et al.  Selective Sampling for Binary Choice Models , 2003 .

[37]  Katherine N. Lemon,et al.  Dynamic Customer Relationship Management: Incorporating Future Considerations into the Service Retention Decision , 2002 .

[38]  Eric W. T. Ngai,et al.  Customer churn prediction using improved balanced random forests , 2009, Expert Syst. Appl..

[39]  John O. Summers,et al.  Relationship Marketing Activities, Commitment, and Membership Behaviors in Professional Associations , 2000 .

[40]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[41]  Euiho Suh,et al.  An LTV model and customer segmentation based on customer value: a case study on the wireless telecommunication industry , 2004, Expert Syst. Appl..

[42]  R. Brodie,et al.  Building models for marketing decisions , 2000 .

[43]  Phillip E. Pfeifer,et al.  The elasticity of customer value to retention: The duration of a customer relationship , 2004 .

[44]  Nissan Levin,et al.  Predictive modeling using segmentation , 2001 .

[45]  R. Rust,et al.  Customer satisfaction, customer retention, and market share , 1993 .

[46]  X. Drèze,et al.  An empirical investigation of the impact of communication timing on customer equity , 2008 .

[47]  Wagner A. Kamakura,et al.  Defection Detection: Measuring and Understanding the Predictive Accuracy of Customer Churn Models , 2006 .

[48]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[49]  V. Mittal,et al.  Satisfaction, Repurchase Intent, and Repurchase Behavior: Investigating the Moderating Effect of Customer Characteristics , 2001 .

[50]  Sunil Gupta,et al.  Customer-Based Valuation , 2009 .