A Unified Approach to the Extraction of Rules from Artificial Neural Networks and Support Vector Machines

Support Vector Machines (SVM) are believed to be as powerful as Artificial Neural Networks (ANN) in modeling complex problems while avoiding some of the drawbacks of the latter such as local minimae or reliance on architecture. However, a question that remains to be answered is whether SVM users may expect improvements in the interpretability of their models, namely by using rule extraction methods already available to ANN users. This study successfully applies the Orthogonal Search-based Rule Extraction algorithm (OSRE) to Support Vector Machines. The study evidences the portability of rules extracted using OSRE, showing that, in the case of SVM, extracted rules are as accurate and consistent as those from equivalent ANN models. Importantly, the study also shows that the OSRE method benefits from SVM specific characteristics, being able to extract less rules from SVM than from equivalent ANN models.

[1]  Sholom M. Weiss,et al.  An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods , 1989, IJCAI.

[2]  Derong Liu,et al.  Advances in Neural Networks - ISNN 2007, 4th International Symposium on Neural Networks, ISNN 2007, Nanjing, China, June 3-7, 2007, Proceedings, Part I , 2007, ISNN.

[3]  David J. Crisp,et al.  Uniqueness of the SVM Solution , 1999, NIPS.

[4]  Terence Anthony Etchells.  Rule extraction from neural networks : a practical and efficient approach , 2003 .

[5]  Paulo J. G. Lisboa,et al.  A Prototype Integrated Decision Support System for Breast Cancer Oncology , 2007, IWANN.

[6]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[7]  Raymond J. Mooney,et al.  Symbolic and neural learning algorithms: An experimental comparison , 1991, Machine Learning.

[8]  Paulo J. G. Lisboa,et al.  Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach , 2006, IEEE Transactions on Neural Networks.

[9]  Alister Hamilton,et al.  9th International Work-Conference on Artificial Neural Networks , 2007 .

[10]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[11]  Rudy Setiono,et al.  Extracting -of- Rules from Trained Neural Networks , 2000 .

[12]  Douglas H. Fisher,et al.  An Empirical Comparison of ID3 and Back-propagation , 1989, IJCAI.

[13]  Paulo J. G. Lisboa,et al.  Time-to-event analysis with artificial neural networks: An integrated analytical and rule-based study for breast cancer , 2008, Neural Networks.

[15]  Paulo J. G. Lisboa,et al.  An integrated framework for risk profiling of breast cancer patients following surgery , 2008, Artif. Intell. Medicine.

[16]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[17]  Sabine Van Huffel,et al.  Comparing Analytical Decision Support Models Through Boolean Rule Extraction: A Case Study of Ovarian Tumour Malignancy , 2007, ISNN.