Extracting comprehensible rules from neural networks via genetic algorithms

A common problem in KDD (Knowledge Discovery in Databases) is the presence of noise in the data being mined. Neural networks are robust and have a good tolerance to noise, which makes them suitable for mining very noisy data. However, they have the well-known disadvantage of not discovering any high-level rule that can be used as a support for human decision making. In this work we present a method for extracting accurate, comprehensible rules from neural networks. The proposed method uses a genetic algorithm to find a good neural network topology. This topology is then passed to a rule extraction algorithm, and the quality of the extracted rules is then fed back to the genetic algorithm. The proposed system is evaluated on three public-domain data sets and the results show that the approach is valid.

[1]  Renu Vig,et al.  Medical Diagnostic Expert System , 1993, HCI.

[2]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[3]  Michael C. Mozer,et al.  The Connectionist Scientist Game: Rule Extraction and Refinement in a Neural Network , 1991 .

[4]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[5]  J. William Ahwood,et al.  CLASSIFICATION , 1931, Foundations of Familiar Language.

[6]  Hongjun Lu,et al.  NeuroRule: A Connectionist Approach to Data Mining , 1995, VLDB.

[7]  Ron Kohavi,et al.  Error-Based and Entropy-Based Discretization of Continuous Features , 1996, KDD.

[8]  Yoichi Hayashi,et al.  A neural expert system using fuzzy teaching input , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[9]  Jude W. Shavlik,et al.  in Advances in Neural Information Processing , 1996 .

[10]  Yoshiki Uchikawa,et al.  On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm , 1992, IEEE Trans. Neural Networks.

[11]  LiMin Fu,et al.  Neural networks in computer intelligence , 1994 .

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[14]  Martin Mandischer,et al.  Representation and Evolution of Neural Networks , 1993 .

[15]  Eric R. Ziegel,et al.  Neural Networks in Computer Intelligence , 1995 .

[16]  R. Nakano,et al.  Medical diagnostic expert system based on PDP model , 1988, IEEE 1988 International Conference on Neural Networks.

[17]  David W. Opitz,et al.  Using Genetic Search to Refine Knowledge-based Neural Networks , 1994, ICML.

[18]  Zhengjun Pan,et al.  Evolving both the Topology and Weights of Neural Networks* , 1996, Parallel Algorithms Appl..

[19]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[20]  Jason Catlett,et al.  Overprvning Large Decision Trees , 1991, IJCAI.

[21]  Hamid R. Berenji,et al.  Refinement of Approximate Reasoning-based Controllers by Reinforcement Learning , 1991, ML.

[22]  Huan Liu,et al.  Book review: Machine Learning, Neural and Statistical Classification Edited by D. Michie, D.J. Spiegelhalter and C.C. Taylor (Ellis Horwood Limited, 1994) , 1996, SGAR.

[23]  David W. Aha,et al.  Simplifying decision trees: A survey , 1997, The Knowledge Engineering Review.

[24]  Jude Shavlik,et al.  THE EXTRACTION OF REFINED RULES FROM KNOWLEDGE BASED NEURAL NETWORKS , 1993 .

[25]  James J. Buckley,et al.  Approximations between fuzzy expert systems and neural networks , 1994, Int. J. Approx. Reason..

[26]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[27]  David J. Hand,et al.  Construction and Assessment of Classification Rules , 1997 .

[28]  LiMin Fu,et al.  Rule Learning by Searching on Adapted Nets , 1991, AAAI.

[29]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery: An Overview , 1996, Advances in Knowledge Discovery and Data Mining.