QSVM: A Support Vector Machine for Rule Extraction

Rule extraction from neural networks represents a difficult research problem, which is NP-hard. In this work we show how a special Multi Layer Perceptron architecture denoted as DIMLP can be used to extract rules from ensembles of DIMLPs and Quantized Support Vector Machines (QSVMs). The key idea for rule extraction is that the locations of discriminative hyperplanes are known, precisely. Based on ten repetitions of stratified 10-fold cross validation trials and with the use of default learning parameters we generated symbolic rules from five datasets. The obtained results compared favorably with respect to another state of the art technique applied to Support Vector Machines.

[1]  Andreu Català,et al.  Rule extraction from support vector machines , 2002, ESANN.

[2]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  Zhi-Hua Zhou,et al.  Extracting symbolic rules from trained neural network ensembles , 2003, AI Commun..

[5]  Andreu Català,et al.  Rule-Based Learning Systems for Support Vector Machines , 2006, Neural Processing Letters.

[6]  Sushmita Mitra,et al.  A new approach to three ensemble neural network rule extraction using recursive-rule extraction algorithm , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[7]  Andrew P. Bradley,et al.  Rule extraction from support vector machines: A review , 2010, Neurocomputing.

[8]  Stephen I. Gallant,et al.  Connectionist expert systems , 1988, CACM.

[9]  Guido Bologna,et al.  A Study on Rule Extraction from Several Combined Neural Networks , 2001, Int. J. Neural Syst..

[10]  Yoichi Hayashi,et al.  Ensemble neural network rule extraction using Re-RX algorithm , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[11]  Guido Bologna,et al.  Is it worth generating rules from neural network ensembles? , 2004, J. Appl. Log..

[12]  Wlodzislaw Duch,et al.  A new methodology of extraction, optimization and application of crisp and fuzzy logical rules , 2001, IEEE Trans. Neural Networks.

[13]  Guido Bologna,et al.  Intelligent Assistance for Coronary Heart Disease Diagnosis: A Comparison Study , 1997, AIME.

[14]  Bart Baesens,et al.  Recursive Neural Network Rule Extraction for Data With Mixed Attributes , 2008, IEEE Transactions on Neural Networks.

[15]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[16]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[17]  Joachim Diederich,et al.  Rule Extraction from Support Vector Machines , 2008, Studies in Computational Intelligence.