Class balancing in customer segments classification using support vector machine rule extraction and ensemble learning

An objective and data-based market segmentation is a precondition for efficient targeting in direct marketing campaigns. The role of customer segments classification in direct marketing is to predict the segment of most valuable customers who is likely to respond to a campaign based on previous purchasing behavior. A good-performing predictive model can significantly increase revenue, but also, reduce unnecessary marketing campaign costs. As this segment of customers is generally the smallest, most classification methods lead to misclassification of the minor class. To overcome this problem, this paper proposes a class balancing approach based on Support Vector Machine-Rule Extraction (SVM-RE) and ensemble learning. Additionally, this approach allows for rule extraction, which can describe and explain different customer segments. Using a customer base from a company’s direct marketing campaigns, the proposed approach is compared to other data balancing methods in terms of overall prediction accuracy, recall and precision for the minor class, as well as profitability of the campaign. It was found that the method performs better than other compared class balancing methods in terms of all mentioned criteria. Finally, the results confirm the superiority of the ensemble SVM method as a preprocessor, which effectively balances data in the process of customer segments classification.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  Farshid Abdi,et al.  Hybrid soft computing approach based on clustering, rule mining, and decision tree analysis for customer segmentation problem: Real case of customer-centric industries , 2018, Appl. Soft Comput..

[3]  Stefan Lessmann,et al.  Targeting customers for profit: An ensemble learning framework to support marketing decision-making , 2019, Inf. Sci..

[4]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[5]  Bart Baesens,et al.  Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines , 2007, Eur. J. Oper. Res..

[6]  Enrico Zio,et al.  Integration of feature vector selection and support vector machine for classification of imbalanced data , 2019, Appl. Soft Comput..

[7]  T. Wansbeek,et al.  Optimal selection of households for direct marketing by joint modeling of the probability and quantity of response , 2006 .

[8]  C.-Y. Tsai,et al.  A purchase-based market segmentation methodology , 2004, Expert Syst. Appl..

[9]  Miomir Jovanovic,et al.  Hybrid support vector machine rule extraction method for discovering the preferences of stock market investors: Evidence from Montenegro , 2015, Intell. Autom. Soft Comput..

[10]  Ede Lázár Customer Churn Prediction Embedded in an Analytical CRM Model , 2015 .

[11]  Xi Chen,et al.  Quantitative models for direct marketing: A review from systems perspective , 2009, Eur. J. Oper. Res..

[12]  Eneko Osaba,et al.  Ensemble classification for imbalanced data based on feature space partitioning and hybrid metaheuristics , 2019, Applied Intelligence.

[13]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[15]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[16]  David L. Olson,et al.  A support vector machine (SVM) approach to imbalanced datasets of customer responses: comparison with other customer response models , 2012, Service Business.

[17]  Bob Stone,et al.  Successful Direct Marketing Methods , 1975 .

[18]  Joachim Diederich,et al.  Rule Extraction from Support Vector Machines: An Introduction , 2008, Rule Extraction from Support Vector Machines.

[19]  Dirk Van den Poel,et al.  Joint optimization of customer segmentation and marketing policy to maximize long-term profitability , 2002, Expert Syst. Appl..

[20]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[21]  Hidayet Takçi,et al.  Performance evaluation of different customer segmentation approaches based on RFM and demographics analysis , 2016, Kybernetes.

[22]  Dalia Susa Vugec,et al.  Understanding impact of business intelligence to organizational performance using cluster analysis: does culture matter? , 2018, International Journal of Information Systems and Project Management.

[23]  You-Shyang Chen,et al.  Classifying the segmentation of customer value via RFM model and RS theory , 2009, Expert Syst. Appl..

[24]  Bart Baesens,et al.  Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring , 2008, Rule Extraction from Support Vector Machines.

[25]  Bart Baesens,et al.  Building comprehensible customer churn prediction models with advanced rule induction techniques , 2011, Expert Syst. Appl..

[26]  David J. Curry,et al.  Prediction in Marketing Using the Support Vector Machine , 2005 .

[27]  Andrew K. C. Wong,et al.  Classification of Imbalanced Data: a Review , 2009, Int. J. Pattern Recognit. Artif. Intell..

[28]  Seyed Mohammad Seyedhosseini,et al.  Cluster analysis using data mining approach to develop CRM methodology to assess the customer loyalty , 2010, Expert Syst. Appl..

[29]  Ksenija Dumicic,et al.  Exploratory study of insurance companies in selected post-transition countries: non-hierarchical cluster analysis , 2018, Central Eur. J. Oper. Res..

[30]  Gavin Brown,et al.  Ensemble Learning , 2010, Encyclopedia of Machine Learning and Data Mining.

[31]  Man Leung Wong,et al.  Targeting High Value Customers While under Resource Constraint: Partial Order Constrained Optimization with Genetic Algorithm , 2015 .

[32]  A. Hughes Strategic Database Marketing: The Masterplan for Starting and Managing a Profitable, Customer-Based Marketing Program , 1994 .

[33]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[34]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[35]  Man Leung Wong,et al.  Cost-sensitive ensemble of stacked denoising autoencoders for class imbalance problems in business domain , 2020, Expert Syst. Appl..

[36]  U. Kaymak Fuzzy target selection using RFM variables , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[37]  Zou Peng,et al.  Customer value segmentation based on cost-sensitive learning Support Vector Machine , 2010, Int. J. Serv. Technol. Manag..

[38]  M. A. H. Farquad,et al.  Preprocessing unbalanced data using support vector machine , 2012, Decis. Support Syst..

[39]  Vladimir Djurisic,et al.  Bank CRM Optimization Using Predictive Classification Based on the Support Vector Machine Method , 2020, Appl. Artif. Intell..

[40]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[41]  Hyoungjoo Lee,et al.  Response modeling with support vector regression , 2008, Expert Syst. Appl..

[42]  Edward Y. Chang,et al.  Class-Boundary Alignment for Imbalanced Dataset Learning , 2003 .

[43]  Ravi Shankar,et al.  A Computational Intelligence based Approach to Telecom Customer Classification for Value Added Services , 2012, BIC-TA.

[44]  Chih-Hsuan Wang,et al.  Apply robust segmentation to the service industry using kernel induced fuzzy clustering techniques , 2010, Expert Syst. Appl..

[45]  Armin Lawi,et al.  On identifying potential direct marketing consumers using adaptive boosted support vector machine , 2017, 2017 4th International Conference on Computer Applications and Information Processing Technology (CAIPT).

[46]  Nan-Chen Hsieh,et al.  An integrated data mining and behavioral scoring model for analyzing bank customers , 2004, Expert Syst. Appl..

[47]  David L. Olson,et al.  Comparison of customer response models , 2009 .

[48]  Andrew P. Bradley,et al.  Rule extraction from support vector machines: A review , 2010, Neurocomputing.

[49]  The Effects of Digitalization on Customer Experience , 2019, SSRN Electronic Journal.

[50]  Sungzoon Cho,et al.  Improved response modeling based on clustering, under-sampling, and ensemble , 2012, Expert Syst. Appl..

[51]  Michael R. Lyu,et al.  Learning classifiers from imbalanced data based on biased minimax probability machine , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[52]  Martina Tomičić Furjan,et al.  Understanding Digital Transformation Initiatives: Case Studies Analysis , 2020, Business Systems Research Journal.

[53]  P. Danaher,et al.  Implementing a customer relationship strategy: The asymmetric impact of poor versus excellent execution , 2000 .

[54]  Garima Gupta,et al.  Comparative Study of Random Forest and Neural Network for Prediction in Direct Marketing , 2018, Advances in Intelligent Systems and Computing.

[55]  Ksenija Dumičić,et al.  Business Client Segmentation in Banking Using Self-Organizing Maps , 2013 .

[56]  David L. Olson,et al.  Direct marketing decision support through predictive customer response modeling , 2012, Decis. Support Syst..

[57]  Iraklis A. Klampanos Manning Christopher, Prabhakar Raghavan, Hinrich Schütze: Introduction to information retrieval , 2009, Information Retrieval.

[58]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[59]  Suncica Rogic,et al.  Customer Value Prediction in Direct Marketing Using Hybrid Support Vector Machine Rule Extraction Method , 2019, ADBIS.

[60]  Edward C. Malthouse,et al.  Ridge regression and direct marketing scoring models , 1999 .

[61]  M. M. al-Rifaie,et al.  Handling class imbalance in direct marketing dataset using a hybrid data and algorithmic level solutions , 2016, 2016 SAI Computing Conference (SAI).

[62]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[63]  Sophia Daskalaki,et al.  Imbalanced customer classification for bank direct marketing , 2017, Journal of Marketing Analytics.

[64]  Ana S. Camanho,et al.  Predicting direct marketing response in banking: comparison of class imbalance methods , 2017 .

[65]  John A. McCarty,et al.  SEGMENTATION APPROACHES IN DATA MINING: A COMPARISON OF RFM, CHAID, AND LOGISTIC REGRESSION , 2007 .