PSO-based approach for neural network ensembles

To improve the prediction accuracy of model represented by artificial neural networks(ANN) ensemble, a new approach for constructing neural network ensemble was proposed. The basic idea of this novel approach was to optimally construct neural network ensemble with the aid of particle swarm optimizations (PSO). After a number of neural networks (NN) were properly trained, all possible NN ensembles were represented by particles in a multi-dimensional space, in which each dimension represented a particular NN, and the value 0 or 1 for each dimension meant whether or not the corresponding NN was to be included in the ensemble. Then discrete PSO algorithm was used to optimally select the ensembles. The results of the selection caused the optimized ensemble to include individual NN members with higher diversity. The prediction error of the model expressed by ensemble was determined by the degree of correlation between NN components, which was also used as the fitness function. Empirical studies on regression upon eight typical data sets show that this approach yields ensemble with significantly smaller size, while achieving much better performance than other traditional ones such as Bagging method.