Learning Capability: Classical RBF Network vs. SVM with Gaussian Kernel

The Support Vector Machine (SVM) has recently been introduced as a new learning technique for solving variety of real-world applications based on statistical learning theory. The classical Radial Basis Function (RBF) network has similar structure as SVM with Gaussian kernel. In this paper we have compared the generalization performance of RBF network and SVM in classification problems. We applied Lagrangian differential gradient method for training and pruning RBF network. RBF network shows better generalization performance and computationally faster than SVM with Gaussian kernel, specially for large training data sets.

[1]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[2]  Emanuela Merelli,et al.  A successive overrelaxation backpropagation algorithm for neural-network training , 1998, IEEE Trans. Neural Networks.

[3]  Terrence L. Fine,et al.  Parameter Convergence and Learning Curves for Neural Networks , 1999, Neural Computation.

[4]  George W. Irwin,et al.  A hybrid linear/nonlinear training algorithm for feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[5]  Giovanna Castellano,et al.  An iterative pruning algorithm for feedforward neural networks , 1997, IEEE Trans. Neural Networks.

[6]  Tom Heskes,et al.  On Natural Learning and Pruning in Multilayered Perceptrons , 2000, Neural Computation.

[7]  Mingui Sun,et al.  An adaptive training algorithm for back-propagation neural networks , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[8]  David A. Cohn,et al.  Separating formal bounds from practical performance in learning systems , 1992 .

[9]  K. C. Ho,et al.  A formal selection and pruning algorithm for feedforward artificial neural network optimization , 1999, IEEE Trans. Neural Networks.

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[12]  Dimitris A. Karras,et al.  An efficient constrained training algorithm for feedforward networks , 1995, IEEE Trans. Neural Networks.

[13]  Maher A. Sid-Ahmed,et al.  A new scheme for training feed-forward neural networks , 1997, Pattern Recognit..

[14]  Luigi Grippo,et al.  Convergent Decomposition Techniques for Training RBF Neural Networks , 2001, Neural Computation.

[15]  Ji Zhu,et al.  Kernel Logistic Regression and the Import Vector Machine , 2001, NIPS.

[16]  J. Nazuno Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .