Comparison of Diverse Ensemble Neural Network for Large Data Classification

In a large dataset classification, a higher number of attributes commonly evolve over time, where many dynamic learning strategies have been proposed such as the ensemble network and incremental neural network. Ensemble network is a learning paradigm where many neural networks are jointly used to solve a problem. The relationship between the ensemble and component of neural networks is analyzed from the context of classification in integrated framework. This task would reveal that, it may be better to have many neural networks instead the incremental neural network. Most approaches of ensemble using totally different classifiers for prediction. Then, in order to find an appropriate neural network from ensemble members, it can be selected from a set of different available neural networks. Thus, a Distributed Reordering Technique (DRT) is proposed. DRT is an enhanced algorithm based on distributed random for different neural networks. The weights are randomly assigned to networks in order to evolve, so that they can characterize each neural network to some extent of fitness in constituting a better result. The ensemble network integrated framework supported by the selection of some neural networks based on output and weights that made up the ensemble. The experimental study shows that in comparing with some ensemble approaches such as Bagging, DRT can generate a neural network with enhanced performance and stronger generalization ability. Furthermore, the use of DRT for neural network classifier is practical and relevance to classification systems for large and can be applied to different large data dimension in future.

[1]  Juan José Rodríguez Diez,et al.  A weighted voting framework for classifiers ensembles , 2012, Knowledge and Information Systems.

[2]  Hadi Sadoghi Yazdi,et al.  Making Diversity Enhancement Based on Multiple Classifier System by Weight Tuning , 2012, Neural Processing Letters.

[3]  Giorgio Valentini,et al.  Simple ensemble methods are competitive with state-of-the-art data integration methods for gene function prediction , 2010, MLSB.

[4]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[5]  Jie Zhang,et al.  Selective combination of multiple neural networks for improving model prediction in nonlinear systems modelling through forward selection and backward elimination , 2009, Neurocomputing.

[6]  Yann LeCun,et al.  Large Scale Online Learning , 2003, NIPS.

[7]  Zhi-Hua Zhou,et al.  Multi-information Ensemble Diversity , 2010, MCS.

[8]  Jing Shi,et al.  Bayesian adaptive combination of short-term wind speed forecasts from neural network models , 2011 .

[9]  Ethem Alpaydin,et al.  Incremental construction of classifier and discriminant ensembles , 2009, Inf. Sci..

[10]  Hamid Parvin,et al.  To improve the quality of cluster ensembles by selecting a subset of base clusters , 2014, J. Exp. Theor. Artif. Intell..

[11]  Xin Yao,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Relationships between Diversity of Classification Ensembles and Single-class Performance Measures , 2022 .

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Héctor Allende,et al.  Behavior analysis of neural network ensemble algorithm on a virtual machine cluster , 2011, Neural Computing and Applications.

[14]  Qi Chen,et al.  An improved incremental learning model for network data stream classification problems , 2012 .

[15]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Xiaoou Li,et al.  Support vector machine classification for large data sets via minimum enclosing ball clustering , 2008, Neurocomputing.

[17]  William F. Punch,et al.  Effects of resampling method and adaptation on clustering ensemble efficacy , 2011, Artificial Intelligence Review.

[18]  Johannes R. Sveinsson,et al.  A classifier ensemble based on fusion of support vector machines for classifying hyperspectral data , 2010 .

[19]  Sotiris B. Kotsiantis,et al.  An incremental ensemble of classifiers , 2011, Artificial Intelligence Review.

[20]  Emilio Corchado,et al.  A survey of multiple classifier systems as hybrid systems , 2014, Inf. Fusion.

[21]  Liying Yang,et al.  Classifiers selection for ensemble learning based on accuracy and diversity , 2011 .

[22]  Joaquín Torres-Sospedra,et al.  Introducing Reordering Algorithms to Classic Well-Known Ensembles to Improve Their Performance , 2011, ICONIP.

[23]  M. Mohamad,et al.  The Use of Output Combiners in Enhancing the Performance of Large Data for ANNs , 2014 .

[24]  E. T. Senalp,et al.  The neural network technique--1: a general exposition , 2004 .

[25]  R. Ebrahimpour,et al.  Improving ECG Classification Accuracy Using an Ensemble of Neural Network Modules , 2011, PloS one.

[26]  Faliang Huang,et al.  Research on Ensemble Learning , 2009, 2009 International Conference on Artificial Intelligence and Computational Intelligence.

[27]  Kin Keung Lai,et al.  Credit risk assessment with a multistage neural network ensemble learning approach , 2008, Expert Syst. Appl..

[28]  Payam Hanafizadeh,et al.  Ensemble strategies to build neural network to facilitate decision making , 2008 .

[29]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[30]  Terry Windeatt,et al.  Ensemble MLP Classifier Design , 2008, Computational Intelligence Paradigms.

[31]  Yang Yang,et al.  A neural network ensemble method with new definition of diversity based on output error curve , 2010, 2010 International Conference on Intelligent Computing and Integrated Systems.