Training of neural network ensemble through progressive interaction

This paper presents an interactive training method for neural network ensembles (NNEs). For an NNE, proposed method trains component neural networks (NNs) one after another sequentially and interactions among the NNs are maintained indirectly via an intermediate space, called information center (IC). IC manages outputs of all previously trained NNs. Update rule, to train an NN in conjunction with IC, is developed from negative correlation learning (NCL) and defined the proposed method as progressive NCL (pNCL). The introduction of such an information center in ensemble methods reduces the training time interaction among component NNs. The effectiveness of the proposed method is evaluated on several benchmark classification problems. The experimental results show that the proposed approach can improve the performance of NNEs. pNCL is incorporated with two popular NNE methods, bagging and boosting. It is also found that the performance of bagging and boosting algorithms can be further improved by incorporating pNCL with their training processes.

[1]  T. Stützle,et al.  A Review on the Ant Colony Optimization Metaheuristic: Basis, Models and New Trends , 2002 .

[2]  Jiwu Shu,et al.  A Cluster LVM for SAN Environments , 2005, ISPA Workshops.

[3]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[4]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[5]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[6]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[7]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[8]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[9]  Xin Yao,et al.  Simultaneous training of negatively correlated neural networks in an ensemble , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[11]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[12]  Amanda J. C. Sharkey,et al.  On Combining Artificial Neural Nets , 1996, Connect. Sci..

[13]  Noel E. Sharkey,et al.  Combining diverse neural nets , 1997, The Knowledge Engineering Review.

[14]  Mykola Pechenizkiy,et al.  Diversity in search strategies for ensemble feature selection , 2005, Inf. Fusion.

[15]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .