Data Mining Using Rules Extracted from SVM: An Application to Churn Prediction in Bank Credit Cards

In this work, an eclectic procedure for rule extraction from Support Vector Machine is proposed, where Tree is generated using Naive Bayes Tree (NBTree) resulting in the SVM+NBTree hybrid. The data set analyzed in this paper is about churn prediction in bank credit cards and is obtained from Business Intelligence Cup 2004. The data set under consideration is highly unbalanced with 93.11% loyal and 6.89% churned customers. Since identifying churner is of paramount importance from business perspective, sensitivity of classification model is more critical. Using the available, original unbalanced data only, we observed that the proposed hybrid SVM+NBTree yielded the best sensitivity compared to other classifiers.

[1]  Andrew P. Bradley,et al.  Rule Extraction from Support Vector Machines: Measuring the Explanation Capability Using the Area under the ROC Curve , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  Ying Zhang,et al.  Rule Extraction from Trained Support Vector Machines , 2005, PAKDD.

[3]  Ted E. Senator,et al.  The Financial Crimes Enforcement Network AI System (FAIS) Identifying Potential Money Laundering from Reports of Large Cash Transactions , 1995, AI Mag..

[4]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[5]  V. Ravi,et al.  Rule extraction using Support Vector Machine based hybrid classifier , 2008, TENCON 2008 - 2008 IEEE Region 10 Conference.

[6]  Joachim Diederich,et al.  Learning-Based Rule-Extraction From Support Vector Machines: Performance On Benchmark Data Sets , 2004 .

[7]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[8]  Stephen I. Gallant,et al.  Connectionist expert systems , 1988, CACM.

[9]  Ricardo Tanscheit,et al.  Fuzzy rule extraction from support vector machines , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[10]  Xin Yao,et al.  A novel evolutionary data mining algorithm with applications to churn prediction , 2003, IEEE Trans. Evol. Comput..

[11]  Pavel Brazdil,et al.  Proceedings of the European Conference on Machine Learning , 1993 .

[12]  N. Mols,et al.  The behavioral consequences of PC banking , 1998 .

[13]  Andrew P. Bradley,et al.  Rule Extraction from Support Vector Machines: A Sequential Covering Approach , 2007, IEEE Transactions on Knowledge and Data Engineering.

[14]  Vadlamani Ravi,et al.  Support Vector Machine based Hybrid Classifiers and Rule Extraction thereof: Application to Bankruptcy Prediction in Banks , 2010 .

[15]  P. K. Kannan,et al.  Implications of loyalty program membership and service experiences for customer retention and value , 2000 .

[16]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[17]  Emilio Soria Olivas,et al.  Handbook of Research on Machine Learning Applications and Trends : Algorithms , Methods , and Techniques , 2009 .

[18]  Glenn Fung,et al.  Rule extraction from linear support vector machines , 2005, KDD '05.

[19]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[20]  Miguel A. P. M. Lejeune,et al.  Measuring the impact of data mining on churn management , 2001, Internet Res..

[21]  Joachim Diederich,et al.  Eclectic Rule-Extraction from Support Vector Machines , 2005 .

[22]  Jianping Li,et al.  A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue , 2007, Artif. Intell. Medicine.

[23]  Dirk Van den Poel,et al.  Predicting customer retention and profitability by using random forests and regression forests techniques , 2005, Expert Syst. Appl..

[24]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[25]  V. Ravi,et al.  Sputter process variables prediction via data mining , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[26]  Vadlamani Ravi,et al.  Predicting credit card customer churn in banks using data mining , 2008, Int. J. Data Anal. Tech. Strateg..

[27]  Vadlamani Ravi,et al.  Application of fuzzyARTMAP for churn prediction in bank credit cards , 2009, Int. J. Inf. Decis. Sci..

[28]  Bart Baesens,et al.  Decompositional Rule Extraction from Support Vector Machines by Active Learning , 2009, IEEE Transactions on Knowledge and Data Engineering.

[29]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[30]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[31]  Cheng-Seen Ho,et al.  Toward a hybrid data mining model for customer retention , 2007, Knowl. Based Syst..

[32]  Ruth N. Bolton,et al.  A Dynamic Model of the Duration of the Customer's Relationship with a Continuous Service Provider: The Role of Satisfaction , 1994 .

[33]  Andreu Català,et al.  Rule extraction from support vector machines , 2002, ESANN.