A comparison of combination methods for ensembles of RBF networks

Building an ensemble of classifiers is an useful way to improve the performance. In the case of neural networks the bibliography has centered on the use of multilayer feedforward (MF). However, there are other interesting networks like radial basis functions (RBF) that can be used as elements of the ensemble. In a previous paper we presented results of different methods to build the ensemble of RBF. The results showed that the best method is in general the simple ensemble. The combination methods used in that research was averaging. In this paper we present results of fourteen different combination methods for a simple ensemble of RBF. The best performing methods are Borda count, weighted average and majority voting.

[1]  H. Zimmermann,et al.  Decisions and evaluations by hierarchical aggregation of information , 1983 .

[2]  Mercedes Fernández-Redondo,et al.  Ensembles of RBFs Trained by Gradient Descent , 2004, ISNN.

[3]  D. Jimenez,et al.  Dynamically weighted ensemble neural networks for classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[4]  Antanas Verikas,et al.  Soft combination of neural classifiers: A comparative study , 1999, Pattern Recognit. Lett..

[5]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[6]  Nicolaos B. Karayiannis,et al.  On the construction and training of reformulated radial basis function neural networks , 2003, IEEE Trans. Neural Networks.

[7]  Kagan Tumer,et al.  Error Correlation and Error Reduction in Ensemble Classifiers , 1996, Connect. Sci..

[8]  Mohamed S. Kamel,et al.  Decision fusion in neural network ensembles , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[9]  Nathan Intrator,et al.  Bootstrapping with Noise: An Effective Regularization Technique , 1996, Connect. Sci..

[10]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.