Neural Network Rule Extraction and the LED Display Recognition Problem

This paper presents the results from a neural network rule extraction algorithm applied to the LED display recognition problem. We show that pruned neural networks with small number of hidden nodes and connections are able to recognize all the 10 digits from 0 to 9. Earlier work by other researchers demonstrated how symbolic fuzzy rules can be extracted from trained neural networks to solve this problem. Our rules in contrast are crisp rules, and they are obtained from smaller networks. As a result, simpler and easier to understand rules are obtained. These rules give us an insight of how neural networks differentiate one digit from the rest in LED display recognition problem.

[1]  Michael Margaliot,et al.  Knowledge Extraction From Neural Networks Using the All-Permutations Fuzzy Rule Base: The LED Display Recognition Problem , 2005, IEEE Transactions on Neural Networks.

[2]  Ishwar K. Sethi,et al.  Symbolic mapping of neurons in feedforward networks , 1996, Pattern Recognit. Lett..

[3]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[4]  Ignacio Requena,et al.  Are artificial neural networks black boxes? , 1997, IEEE Trans. Neural Networks.

[5]  Rudy Setiono,et al.  A Penalty-Function Approach for Pruning Feedforward Neural Networks , 1997, Neural Computation.

[6]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[7]  Michael Margaliot,et al.  Are artificial neural networks white boxes? , 2005, IEEE Transactions on Neural Networks.

[8]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[9]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[10]  Huan Liu,et al.  Chi2: feature selection and discretization of numeric attributes , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[11]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[12]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[13]  Ishwar K. Sethi.  Neural implementation of tree classifiers , 1995, IEEE Trans. Syst. Man Cybern..

[14]  Mukta Paliwal,et al.  Neural networks and statistical techniques: A review of applications , 2009, Expert Syst. Appl..