Neural Network Ensembles Using Clustering Ensemble and Genetic Algorithm

In this paper, a new method for enhancing the performance of Neural Network ensemble is proposed. The main idea of this method is creating diversity for training artificial neural networks (ANNs) using an interesting method which applies clustering ensemble and genetic algorithm. In combinational classifier systems, the more diversity in results of the base classifiers yields to better final performance. Inspiring from the boosting, the diversity of the base classifiers is provided by different train sets for base classifiers. The different train sets are derived from the original train set by adding some of data samples in train set. Finding near optimal sets is implemented using clustering ensemble technique and genetic algorithm. Finally, the majority vote fuses the outputs of trained MLPs on the new train sets from population of the last generation of GA. Experimental results demonstrate the strength of proposed method on three different datasets.

[1]  Morteza Analoui,et al.  CCHR: Combination of Classifiers Using Heuristic Retraining , 2008, 2008 Fourth International Conference on Networked Computing and Advanced Information Management.

[2]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[3]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[4]  William F. Punch,et al.  Using Genetic Algorithms for Data Mining Optimization in an Educational Web-Based System , 2003, GECCO.

[5]  Harris Drucker,et al.  Improving Performance in Neural Networks Using a Boosting Algorithm , 1992, NIPS.

[6]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[7]  David W. Opitz,et al.  Actively Searching for an E(cid:11)ective Neural-Network Ensemble , 1996 .

[8]  L. Shapley,et al.  Optimizing group judgmental accuracy in the presence of interdependencies , 1984 .

[9]  Anil K. Jain,et al.  Large-scale parallel data clustering , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[10]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[11]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[12]  Jitender S. Deogun,et al.  Conceptual clustering in information retrieval , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[13]  Hamid Parvin,et al.  A New Approach to Improve the Vote-Based Classifier Selection , 2008, 2008 Fourth International Conference on Networked Computing and Advanced Information Management.

[14]  Pablo M. Granitto,et al.  Selecting diverse members of neural network ensembles , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[15]  Günther Palm,et al.  Decision templates for the classification of bioacoustic time series , 2003, Inf. Fusion.

[16]  Zoran Obradovic,et al.  Effective pruning of neural network classifier ensembles , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[17]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[18]  Jianxin Wu,et al.  Genetic Algorithm based Selective Neural Network Ensemble , 2001, IJCAI.

[19]  Fu Qiang PSO-based approach for neural network ensembles , 2004 .

[20]  Mahmood Fathy,et al.  Improved Face Detection Using Spatial Histogram Features , 2008, IPCV.

[21]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[23]  Oleksandr Makeyev,et al.  Neural network with ensembles , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[24]  Morteza Analoui,et al.  A Scalable Method for Improving the Performance of Classifiers in Multiclass Applications by Pairwise Classifiers and GA , 2008, 2008 Fourth International Conference on Networked Computing and Advanced Information Management.

[25]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[26]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[27]  Bruce E. Rosen,et al.  Ensemble Learning Using Decorrelated Neural Networks , 1996, Connect. Sci..