Ensembles of diverse neural networks

ii an ensemble with appropriate networks. Existing ensemble methods, in general, train a predefined number of networks and consider all of them for final ensemble. Therefore, performance of an ensemble might be poor when any network performs very badly. On the other hand, ensemble with properly selected networks might always perform better. In this context, an ensemble method is proposed in this thesis that first creates a pool of diverse networks and then apply selection scheme to construct an ensemble with appropriate networks. Three data sampling techniques are considered to create network pool, and two selection schemes are investigated. Based on experimental results on a large number of benchmark problems, the proposed method is found better than other traditional methods with concise ensemble. In this thesis, an ensemble method is also investigated that automatically determines a minimal ensemble architecture for a given problem. To determine minimal architecture, it starts with a single network with a minimal number of hidden units. During training process, it adds additional network(s) with cumulative number(s) of hidden units and the added network specializes in the previously unsolved portion of the input space. Finally all the networks are trained simultaneously to improve the generalization ability. When a single network is shown to achieve acceptable result for a problem, it does not build and returns the single network for the problem. The proposed method is found competitive with existing ensemble methods with minimal architecture when tested on benchmark problems. Further, an indirect communication scheme among the networks is investigated in this study when they are trained for an ensemble and proposed progressive interactive training scheme. In the proposed scheme, networks are trained one after another and interaction among the networks is maintained indirectly via an intermediate space called information center. The idea of using indirect communication is conceived from the communication among biological ants via pheromone. An individual ant decides its travelling path based on existing pheromone on the trail and also it deposits pheromone on its travelling path. The indirect interaction has several benefits over direct interaction scheme of negative correlation learning. The experimental results show that ensembles construction with the proposed training scheme performs well. The idea of indirect interaction is also incorporated with bagging and AdaBoost, and is shown to improve their performance.