Kernel-based online machine learning and support vector reduction

We apply kernel-based machine learning methods to online learning situations, and look at the related requirement of reducing the complexity of the learnt classifier. Online methods are particularly useful in situations which involve streaming data, such as medical or financial applications. We show that the concept of span of support vectors can be used to build a classifier that performs reasonably well while satisfying given space and time constraints, thus making it potentially suitable for such online situations. The span-based heuristic is observed to be effective under stringent memory limits (that is when the number of support vectors a machine can hold is very small).

[1]  Tu Bao Ho,et al.  An efficient method for simplifying support vector machines , 2005, ICML.

[2]  Yoram Singer,et al.  Support Vector Machines on a Budget , 2006, NIPS.

[3]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[4]  S. Sathiya Keerthi,et al.  Building Support Vector Machines with Reduced Classifier Complexity , 2006, J. Mach. Learn. Res..

[5]  Koby Crammer,et al.  Ultraconservative Online Algorithms for Multiclass Problems , 2001, J. Mach. Learn. Res..

[6]  Ingo Steinwart,et al.  Sparseness of Support Vector Machines , 2003, J. Mach. Learn. Res..

[7]  O. Chapelle,et al.  Bounds on error expectation for SVM , 2000 .

[8]  V. Vapnik,et al.  Bounds on Error Expectation for Support Vector Machines , 2000, Neural Computation.

[9]  Bernhard Schölkopf,et al.  Building Sparse Large Margin Classifiers , 2005, ICML.

[10]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[11]  Chih-Jen Lin,et al.  A Simple Decomposition Method for Support Vector Machines , 2002, Machine Learning.

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[14]  Katya Scheinberg,et al.  Efficient SVM Training Using Low-Rank Kernel Representations , 2002, J. Mach. Learn. Res..

[15]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[16]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[17]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[18]  Bernhard Schölkopf,et al.  Improving the Accuracy and Speed of Support Vector Machines , 1996, NIPS.

[19]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[20]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[21]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[22]  Koby Crammer,et al.  Online Classification on a Budget , 2003, NIPS.

[24]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[25]  Jan Wessnitzer,et al.  ESANN'2007 proceedings - European Symposium on Artificial Neural Networks , 2007 .

[26]  Bernhard Schölkopf,et al.  Bounds on Error Expectation for SVM , 2000 .