Rule Extraction from Support Vector Machine Using Modified Active Learning Based Approach: An Application to CRM

Despite superior generalization performance Support vector machines (SVMs) generate black box models. The process of converting such opaque models into transparent model is often regarded as rule extraction. This paper presents a new approach for rule extraction from SVMs using modified active learning based approach (mALBA), to predict churn in bank credit cards. The dataset is obtained from Business Intelligence Cup 2004, which is highly unbalanced with 93% loyal and 7% churned customers' data. Since identifying churner is paramount from business perspective, therefore considering sensitivity alone, the empirical results suggest that the proposed rule extraction approach using mALBA yielded the best sensitivity compared to other classifiers.

[1]  Jing Zhao,et al.  Bank Customer Churn Prediction Based on Support Vector Machine: Taking a Commercial Bank's VIP Customer Churn as the Example , 2008, 2008 4th International Conference on Wireless Communications, Networking and Mobile Computing.

[2]  Glenn Fung,et al.  Rule extraction from linear support vector machines , 2005, KDD '05.

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[5]  Jianping Li,et al.  A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue , 2007, Artif. Intell. Medicine.

[6]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[7]  Kristof Coussement,et al.  Faculteit Economie En Bedrijfskunde Hoveniersberg 24 B-9000 Gent Churn Prediction in Subscription Services: an Application of Support Vector Machines While Comparing Two Parameter-selection Techniques Churn Prediction in Subscription Services: an Application of Support Vector Machines While Comparin , 2022 .

[8]  Ricardo Tanscheit,et al.  Fuzzy rule extraction from support vector machines , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[9]  L. Ryals,et al.  Cross-functional issues in the implementation of relationship marketing through customer relationship management , 2001 .

[10]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[11]  Andreu Català,et al.  Rule extraction from support vector machines , 2002, ESANN.

[12]  Vadlamani Ravi,et al.  Predicting credit card customer churn in banks using data mining , 2008, Int. J. Data Anal. Tech. Strateg..

[13]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[14]  Cheng-Seen Ho,et al.  Toward a hybrid data mining model for customer retention , 2007, Knowl. Based Syst..

[15]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[16]  Edward H. Shortliffe,et al.  Production Rules as a Representation for a Knowledge-Based Consultation Program , 1977, Artif. Intell..

[17]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  N. Mols,et al.  The behavioral consequences of PC banking , 1998 .

[19]  Andrew P. Bradley,et al.  Rule Extraction from Support Vector Machines: A Sequential Covering Approach , 2007, IEEE Transactions on Knowledge and Data Engineering.

[20]  張 毓騰,et al.  APPLYING DATA MINING TO TELECOM CHURN MANAGEMENT , 2009 .

[21]  Kimmo Alajoutsijärvi,et al.  Customer relationships and the small software firm: A framework for understanding challenges faced in marketing , 2000, Inf. Manag..

[22]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[23]  Emilio Soria Olivas,et al.  Handbook of Research on Machine Learning Applications and Trends : Algorithms , Methods , and Techniques , 2009 .

[24]  Miguel A. P. M. Lejeune,et al.  Measuring the impact of data mining on churn management , 2001, Internet Res..

[25]  Ted E. Senator,et al.  The Financial Crimes Enforcement Network AI System (FAIS) Identifying Potential Money Laundering from Reports of Large Cash Transactions , 1995, AI Mag..

[26]  Cao Kang,et al.  Customer Churn Prediction Based on SVM-RFE , 2008, 2008 International Seminar on Business and Information Management.

[27]  V. Ravi,et al.  Sputter process variables prediction via data mining , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[28]  Vadlamani Ravi,et al.  Support vector regression based hybrid rule extraction methods for forecasting , 2010, Expert Syst. Appl..

[29]  Vadlamani Ravi,et al.  Data Mining Using Rules Extracted from SVM: An Application to Churn Prediction in Bank Credit Cards , 2009, RSFDGrC.

[30]  P. K. Kannan,et al.  Implications of loyalty program membership and service experiences for customer retention and value , 2000 .

[31]  Joachim Diederich,et al.  Eclectic Rule-Extraction from Support Vector Machines , 2005 .

[32]  Eric Johnson,et al.  Predicting subscriber dissatisfaction and improving retention in the wireless telecommunications industry , 2000, IEEE Trans. Neural Networks Learn. Syst..

[33]  Vadlamani Ravi,et al.  Support Vector Machine based Hybrid Classifiers and Rule Extraction thereof: Application to Bankruptcy Prediction in Banks , 2010 .

[34]  Yu Zhao,et al.  Customer Churn Prediction Using Improved One-Class Support Vector Machine , 2005, ADMA.

[35]  Pieter Adriaans,et al.  Data mining , 1996 .

[36]  Ying Zhang,et al.  Rule Extraction from Trained Support Vector Machines , 2005, PAKDD.

[37]  Vadlamani Ravi,et al.  Application of fuzzyARTMAP for churn prediction in bank credit cards , 2009, Int. J. Inf. Decis. Sci..

[38]  K. Ruyter,et al.  Investigating drivers of bank loyalty: the complex relationship between image, service quality and satisfaction , 1998 .

[39]  Jane P. Laudon,et al.  Management Information Systems: Managing the Digital Firm , 2010 .

[40]  V. Ravi,et al.  Rule extraction using Support Vector Machine based hybrid classifier , 2008, TENCON 2008 - 2008 IEEE Region 10 Conference.

[41]  Dirk Van den Poel,et al.  Investigating the role of product features in preventing customer churn, by using survival analysis and choice modeling: The case of financial services , 2004, Expert Syst. Appl..

[42]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[43]  David C. Yen,et al.  Applying data mining to telecom churn management , 2006, Expert Syst. Appl..

[44]  Bart Baesens,et al.  Decompositional Rule Extraction from Support Vector Machines by Active Learning , 2009, IEEE Transactions on Knowledge and Data Engineering.

[45]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[46]  M. Tahar Kechadi,et al.  A new feature set with new window techniques for customer churn prediction in land-line telecommunications , 2010, Expert Syst. Appl..

[47]  Jude W. Shavlik,et al.  Using Sampling and Queries to Extract Rules from Trained Neural Networks , 1994, ICML.

[48]  Stephen I. Gallant,et al.  Connectionist expert systems , 1988, CACM.