SVM incremental learning, adaptation and optimization

The objective of machine learning is to identify a model that yields good generalization performance. This involves repeatedly selecting a hypothesis class, searching the hypothesis class by minimizing a given objective function over the model's parameter space, and evaluating the generalization performance of the resulting model. This search can be computationally intensive as training data continuously arrives, or as one needs to tune hyperparameters in the hypothesis class and the objective function. In this paper, we present a framework for exact incremental learning and adaptation of support vector machine (SVM) classifiers. The approach is general and allows one to learn and unlearn individual or multiple examples, adapt the current SVM to changes in regularization and kernel parameters, and evaluate generalization performance through exact leave-one-out error estimation.

[1]  Massimiliano Pontil,et al.  Properties of Support Vector Machines , 1998, Neural Computation.

[2]  Nello Cristianini,et al.  Query Learning with Large Margin Classi ersColin , 2000 .

[3]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[4]  Nello Cristianini,et al.  The Kernel-Adatron Algorithm: A Fast and Simple Learning Procedure for Support Vector Machines , 1998, ICML.

[5]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[6]  David J. Crisp,et al.  Uniqueness of the SVM Solution , 1999, NIPS.

[7]  Nello Cristianini,et al.  Dynamically Adapting Kernels in Support Vector Machines , 1998, NIPS.

[8]  Katya Scheinberg,et al.  Incremental Learning and Selective Sampling via Parametric Optimization Framework for SVM , 2001, NIPS.

[10]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[11]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[12]  John Platt,et al.  Fast training of svms using sequential minimal optimization , 1998 .

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[15]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[16]  Mario Martín Muñoz.  On-line support vector machines for function approximation , 2002 .