Rule extraction algorithm from support vector machines and its application to credit screening

Developing rule extraction algorithms from machine learning techniques such as artificial neural networks and support vector machines (SVMs), which are considered incomprehensible black-box models, is an important topic in current research. This study proposes a rule extraction algorithm from SVMs that uses a kernel-based clustering algorithm to integrate all support vectors and genetic algorithms into extracted rule sets. This study uses measurements of accuracy, sensitivity, specificity, coverage, fidelity and comprehensibility to evaluate the performance of the proposed method on the public credit screening data sets. Results indicate that the proposed method performs better than other rule extraction algorithms. Thus, the proposed algorithm is an essential analysis tool that can be effectively used in data mining fields.

[1]  Bernhard Schölkopf,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[2]  Glenn Fung,et al.  Rule extraction from linear support vector machines , 2005, KDD '05.

[3]  Yanqing Zhang,et al.  A genetic algorithm-based method for feature subset selection , 2008, Soft Comput..

[4]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[5]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[6]  Carlos García-Martínez,et al.  Evaluating a local genetic algorithm as context-independent local search operator for metaheuristics , 2010, Soft Comput..

[7]  Bart Baesens,et al.  Decompositional Rule Extraction from Support Vector Machines by Active Learning , 2009, IEEE Transactions on Knowledge and Data Engineering.

[8]  Bart Baesens,et al.  Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines , 2007, Eur. J. Oper. Res..

[9]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[10]  Tom V. Mathew.  Genetic Algorithm , 2022 .

[11]  Ricardo Tanscheit,et al.  Fuzzy rule extraction from support vector machines , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[12]  I. Dhillon,et al.  A Unified View of Kernel k-means , Spectral Clustering and Graph Cuts , 2004 .

[13]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[14]  Joachim Diederich,et al.  Rule Extraction from Support Vector Machines: An Introduction , 2008, Rule Extraction from Support Vector Machines.

[15]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.

[16]  Bart Baesens,et al.  Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring , 2008, Rule Extraction from Support Vector Machines.

[17]  Naim Dahnoun,et al.  Studies in Computational Intelligence , 2013 .

[18]  Andrew P. Bradley,et al.  Rule extraction from support vector machines: A review , 2010, Neurocomputing.

[19]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Joachim Diederich,et al.  Rule Extraction from Support Vector Machines , 2008, Studies in Computational Intelligence.

[21]  Nostrand Reinhold,et al.  the utility of using the genetic algorithm approach on the problem of Davis, L. (1991), Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York. , 1991 .

[22]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[23]  M. Esmel ElAlami,et al.  Extracting rules from trained neural network using GA for managing E-business , 2004, Appl. Soft Comput..

[24]  Andreu Català,et al.  Rule-Based Learning Systems for Support Vector Machines , 2006, Neural Processing Letters.

[25]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[26]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[27]  Frank Hsu,et al.  Knowledge Discovery , 2014, Encyclopedia of Social Network Analysis and Mining.

[28]  Anshul Mittal,et al.  A GENETIC ALGORITHM , 2010 .

[29]  Geert Wets,et al.  A hybrid system of neural networks and rough sets for road safety performance indicators , 2010, Soft Comput..

[30]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[31]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[32]  Randall S. Sexton,et al.  Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem , 2006, Eur. J. Oper. Res..

[33]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[34]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[35]  Andrew P. Bradley,et al.  Rule Extraction from Support Vector Machines: A Sequential Covering Approach , 2007, IEEE Transactions on Knowledge and Data Engineering.

[36]  Toshinori Munakata,et al.  Knowledge discovery , 1999, Commun. ACM.

[37]  Rudy Setiono,et al.  A note on knowledge discovery using neural networks and its application to credit card screening , 2009, Eur. J. Oper. Res..

[38]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[39]  Esther-Lydia Silva-Ramírez,et al.  Missing value imputation on missing completely at random data using multilayer perceptrons , 2011, Neural Networks.

[40]  H. Levent Akin,et al.  Rule extraction from trained neural networks using genetic algorithms , 1997 .

[41]  Joachim Diederich,et al.  Eclectic Rule-Extraction from Support Vector Machines , 2005 .