A New Neural Data Analysis Approach Using Ensemble Neural Network Rule Extraction

In this paper, we propose the Ensemble-Recursive-Rule eXtraction (E-Re-RX) algorithm, which is a rule extraction method from ensemble neural networks. We demonstrate that the use of ensemble neural networks produces higher recognition accuracy than individual neural networks and the extracted rules are more comprehensible. E-Re-RX algorithm is an effective rule extraction algorithm for dealing with data sets that mix discrete and continuous attributes. In this algorithm, primary rules are generated as well as secondary rules to handleonlythoseinstances that do not satisfy the primary rules, and then these rules are integrated. We show that this reduces the complexity of using multiple neural networks. This method achieves extremely high recognition rates, even with multiclass problems.

[1]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Yoichi Hayashi,et al.  Combining neural network predictions for medical diagnosis , 2002, Comput. Biol. Medicine.

[3]  Ethem Alpaydin.  Multiple neural networks and weighted voting , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[4]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[5]  Rudy Setiono,et al.  A Penalty-Function Approach for Pruning Feedforward Neural Networks , 1997, Neural Computation.

[6]  Bart Baesens,et al.  Recursive Neural Network Rule Extraction for Data With Mixed Attributes , 2008, IEEE Transactions on Neural Networks.

[7]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[8]  Rudy Setiono,et al.  A note on knowledge discovery using neural networks and its application to credit card screening , 2009, Eur. J. Oper. Res..

[9]  Yoichi Hayashi,et al.  Ensemble neural network rule extraction using Re-RX algorithm , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[10]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.