Neural network ensemble with combination weights adaptive modulation

There exist numerous schemes and methods to construct neural network ensembles. The most common aggregation strategy in ensemble learning is the simple average. Furthermore, we might expect that an improvement can be achieved if there is a method by which we may weigh the individual members in the ensemble according to their individual performance during training stage. The idea behind this is to present an architecture that tries to approach this target. In this paper, a new training algorithm that utilizes a feedback mechanism to iteratively improve the classification capability of the ensemble is presented. This algorithm is called combination weights adaptive modulating (CWAM) algorithm. By CWAM, all the individual networks in the ensemble are trained simultaneously and interactively through the evolving combination weights. To evaluate the performance of CWAM, five well-known classification problems were used. Experimental studies have been carried out to show that the generalization ability of CWAM is much better than that of other neural network ensembles in almost all cases.