Rule Extraction from Support Vector Machines: An Introduction

Rule extraction from support vector machines (SVMs) follows in the footsteps of the earlier effort to obtain human-comprehensible rules from artificial neural networks (ANNs) in order to explain "how" a decision was made or "why" a certain result was achieved. Hence, much of the motivation for the field of rule extraction from support vector machines carries over from the now established area of rule extraction from neural networks. This introduction aims at outlining the significance of extracting rules from SVMs and it will investigate in detail what it means to explain the decision-making process of a machine learning system to a human user who may not be an expert on artificial intelligence or the particular application domain. It is natural to refer to both psychology and philosophy in this context because "explanation" refers to the human mind and its effort to understand the world; the traditional area of philosophical endeavours. Hence, the foundations of current efforts to simulate human explanatory reasoning are discussed as are current limitations and opportunities for rule extraction from support vector machines.

[1]  Alison Cawsey,et al.  Explanation and interaction - the computer generation of explanatory dialogues , 1992, ACL-MIT press series in natural language processing.

[2]  Joachim Diederich,et al.  Eclectic Rule-Extraction from Support Vector Machines , 2005 .

[3]  Ben Taskar,et al.  Learning structured prediction models: a large margin approach , 2005, ICML.

[4]  G. Nigel Gilbert,et al.  Explanation and dialogue , 1989, The Knowledge Engineering Review.

[5]  Johanna D. Moore,et al.  A Reactive Approach to Explanation , 1989, IJCAI.

[6]  Jude W. Shavlik,et al.  Using Sampling and Queries to Extract Rules from Trained Neural Networks , 1994, ICML.

[7]  Stephen I. Gallant,et al.  Connectionist expert systems , 1988, CACM.

[8]  Edward H. Shortliffe,et al.  Production Rules as a Representation for a Knowledge-Based Consultation Program , 1977, Artif. Intell..

[9]  Paul Thagard,et al.  The Best Explanation: Criteria for Theory Choice , 1978 .

[10]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[11]  Yao Wang,et al.  A robust and scalable clustering algorithm for mixed type attributes in large database environment , 2001, KDD '01.

[12]  Andreu Català,et al.  Rule extraction from support vector machines , 2002, ESANN.

[13]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[14]  Pavel Berkhin,et al.  A Survey of Clustering Data Mining Techniques , 2006, Grouping Multidimensional Data.

[15]  Ian Witten,et al.  Data Mining , 2000 .

[16]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[17]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[18]  Paul Thagard,et al.  Models of Scientific Explanation , 2008 .