Silicon Support Vector Machine with On-Line Learning

Training of support vector machines (SVMs) amounts to solving a quadratic programming problem over the training data. We present a simple on-line SVM training algorithm of complexity approximately linear in the number of training vectors, and linear in the number of support vectors. The algorithm implements an on-line variant of sequential minimum optimization (SMO) that avoids the need for adjusting select pairs of training coefficients by adjusting the bias term along with the coefficient of the currently presented training vector. The coefficient assignment is a function of the margin returned by the SVM classifier prior to assignment, subject to inequality constraints. The training scheme lends efficiently to dedicated SVM hardware for real-time pattern recognition, implemented using resources already provided for run-time operation. Performance gains are illustrated using the Kerneltron, a massively parallel mixed-signal VLSI processor for kernel-based real-time video recognition.

[1]  Gert Cauwenberghs,et al.  Stochastic Mixed-Signal VLSI Architecture for High-Dimensional Kernel Machines , 2001, NIPS.

[2]  Gert Cauwenberghs,et al.  Charge-mode parallel architecture for vector-matrix multiplication , 2001 .

[3]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[5]  Nello Cristianini,et al.  The Kernel-Adatron Algorithm: A Fast and Simple Learning Procedure for Support Vector Machines , 1998, ICML.

[6]  Volnei A. Pedroni,et al.  Pattern matching and parallel processing with CCD technology , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[7]  Davide Anguita,et al.  Circuital implementation of support vector machines , 1998 .

[8]  Gert Cauwenberghs,et al.  Delta-sigma algorithmic analog-to-digital conversion , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[9]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[10]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[12]  Gert Cauwenberghs,et al.  Charge-mode parallel architecture for matrix-vector multiplication , 2000, Proceedings of the 43rd IEEE Midwest Symposium on Circuits and Systems (Cat.No.CH37144).

[13]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[14]  Bernhard Schölkopf,et al.  Comparing support vector machines with Gaussian kernels to radial basis function classifiers , 1997, IEEE Trans. Signal Process..

[15]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[16]  Amnon Yariv,et al.  A parallel analog CCD/CMOS neural network IC , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[18]  Tomaso A. Poggio,et al.  Pedestrian detection using wavelet templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[20]  M. J. Howes,et al.  Charge-coupled devices and systems , 1979 .

[21]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[22]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.