Support vector perceptrons

Due to their excellent performance, support vector machines (SVMs) are now used extensively in pattern classification applications. In this paper we show that the standard sigmoidal kernel definition lacks the capability to represent the family of perceptrons, and we propose an improved SVM with a sigmoidal kernel called support vector perceptron (SVP). We show by means of both synthetic and real world data sets that the proposed SVP is able to provide very accurate results in many classification problems, providing maximal margin solutions when classes are separable, and also producing very compact architectures comparable to classical multilayer perceptrons.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Fernando Pérez-Cruz,et al.  Convergence of the IRWLS Procedure to the Support Vector Machine Solution , 2005, Neural Computation.

[3]  Yu Hen Hu,et al.  Structural simplification of a feed-forward, multilayer perceptron artificial neural network , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[4]  Joachim Diederich,et al.  Eclectic Rule-Extraction from Support Vector Machines , 2005 .

[5]  S. Delshadpour.  Reduced size multi layer perceptron neural network for human chromosome classification , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[6]  Fernando Pérez-Cruz,et al.  Weighted least squares training of support vector classifiers leading to compact and adaptive schemes , 2001, IEEE Trans. Neural Networks.

[7]  José Luis Rojo-Álvarez,et al.  Fuzzy sigmoid kernel for support vector classifiers , 2004, Neurocomputing.

[8]  Jacek M. Zurada,et al.  Extraction of rules from artificial neural networks for nonlinear regression , 2002, IEEE Trans. Neural Networks.

[9]  Bernhard Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, International Conference on Machine Learning.

[10]  P. Gay,et al.  A Neural Ensemble For Speech Recognition , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[11]  Eros Gian Alessandro Pasero,et al.  Multi-layer perceptron ensembles for pattern recognition: some experiments , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[14]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[15]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[16]  Guido Bologna,et al.  Rule extraction from a multilayer perceptron with staircase activation functions , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[17]  Aníbal R. Figueiras-Vidal,et al.  Growing support vector classifiers with controlled complexity , 2003, Pattern Recognit..