RGANN: An Efficient Algorithm to Extract Rules from ANNs

This paper describes an efficient rule generation algorithm, called rule generation from artificial neural networks (RGANN) to generate symbolic rules from ANNs. Classification rules are sought in many areas from automatic knowledge acquisition to data mining and ANN rule extraction. This is because classification rules possess some attractive features. They are explicit, understandable and verifiable by domain experts, and can be modified, extended and passed on as modular knowledge. A standard three-layer feedforward ANN is the basis of the algorithm. A four-phase training algorithm is proposed for backpropagation learning. Comparing them to the symbolic rules generated by other methods supports explicitness of the generated rules. Generated rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and predictive accuracy. Extensive experimental studies on several benchmarks classification problems, including breast cancer, wine, season, golf-playing, and lenses classification demonstrate the effectiveness of the proposed approach with good generalization ability.

[1]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[2]  Arbee L. P. Chen,et al.  An Efficient Algorithm for Deriving Compact Rules from Databases , 1995, DASFAA.

[3]  LiMin Fu,et al.  Rule Learning by Searching on Adapted Nets , 1991, AAAI.

[4]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[5]  Huan Liu,et al.  X2R: a fast rule generator , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[6]  Rudy Setiono.  Extracting M-of-N rules from trained neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[7]  Rudy Setiono,et al.  Use of a quasi-Newton method in a feedforward neural network construction algorithm , 1995, IEEE Trans. Neural Networks.

[8]  S. M. Kamruzzaman,et al.  Extracting Symbolic Rules for Medical Diagnosis Problem , 2010, ArXiv.

[9]  Rudy Setiono,et al.  Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting , 1997, Neural Computation.

[10]  Jude W. Shavlik,et al.  Using Sampling and Queries to Extract Rules from Trained Neural Networks , 1994, ICML.

[11]  Jihoon Yang,et al.  Constructive Neural-Network Learning Algorithms for Pattern Classification , 2000 .

[12]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[13]  Tomasz Imielinski,et al.  Database Mining: A Performance Perspective , 1993, IEEE Trans. Knowl. Data Eng..

[14]  Youn-Long Lin,et al.  Performance-driven soft-macro clustering and placement by preserving HDL design hierarchy , 1998, ISPD '98.

[15]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[16]  Ashish Darbari,et al.  Rule Extraction from Trained ANN: A Survey , 2000 .

[17]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[18]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[19]  Duc Truong Pham,et al.  RULES: A simple rule extraction system , 1995 .

[20]  Kazuyuki Murase,et al.  A new algorithm to design compact two-hidden-layer artificial neural networks , 2001, Neural Networks.

[21]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[22]  Duc Truong Pham,et al.  An algorithm for automatic rule induction , 1993, Artif. Intell. Eng..

[23]  Jiawei Han,et al.  Efficient and Effective Clustering Methods for Spatial Data Mining , 1994, VLDB.

[24]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[25]  Jacek M. Zurada,et al.  Extraction of rules from artificial neural networks for nonlinear regression , 2002, IEEE Trans. Neural Networks.

[26]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[27]  Wee Kheng Leow,et al.  FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks , 2004, Applied Intelligence.

[28]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[29]  R. Nakano,et al.  Medical diagnostic expert system based on PDP model , 1988, IEEE 1988 International Conference on Neural Networks.

[30]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[31]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[32]  S. M. Kamruzzaman,et al.  Medical diagnosis using neural network , 2010, ArXiv.

[33]  T. Ash,et al.  Dynamic node creation in backpropagation networks , 1989, International 1989 Joint Conference on Neural Networks.

[34]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[35]  S. M. Kamruzzaman,et al.  An Algorithm to Extract Rules from Artificial Neural Networks for Medical Diagnosis Problems , 2010, ArXiv.

[36]  Huan Liu,et al.  Understanding Neural Networks via Rule Extraction , 1995, IJCAI.

[37]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .