Implementation Issues of an Incremental and Decremental SVM

Incremental and decremental processes of training a support vector machine (SVM) resumes to the migration of vectors in and out of the support set along with modifying the associated thresholds. This paper gives an overview of all the boundary conditions implied by vector migration through the incremental / decremental process. The analysis will show that the same procedures, with very slight variations, can be used for both the incremental and decremental learning. The case of vectors with duplicate contribution is also considered. Migration of vectors among sets on decreasing the regularization parameter is given particularly attention. Experimental data show the possibility of modifying this parameter on a large scale, varying it from complete training (overfitting) to a calibrated value.

[1]  Huan Liu,et al.  Handling concept drifts in incremental learning with support vector machines , 1999, KDD '99.

[2]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[3]  James Theiler,et al.  Accurate On-line Support Vector Regression , 2003, Neural Computation.

[4]  Wenjian Wang,et al.  Online prediction model based on support vector machine , 2008, Neurocomputing.

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[6]  Gert Cauwenberghs,et al.  SVM incremental learning, adaptation and optimization , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[7]  Glenn Fung,et al.  Incremental Support Vector Machine Classification , 2002, SDM.

[8]  Tapio Elomaa,et al.  Machine Learning: ECML 2002 , 2002, Lecture Notes in Computer Science.

[9]  Yann LeCun,et al.  Large Scale Online Learning , 2003, NIPS.

[10]  Gerald W. Kimble,et al.  Information and Computer Science , 1975 .

[11]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Stefan Rüping,et al.  Incremental Learning with Support Vector Machines , 2001, ICDM.

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[15]  Klaus-Robert Müller,et al.  Incremental Support Vector Learning: Analysis, Implementation and Applications , 2006, J. Mach. Learn. Res..

[16]  Liva Ralaivola,et al.  Incremental Learning Algorithms for Classification and Regression: local strategies , 2002 .

[17]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[18]  Thorsten Joachims,et al.  Learning to classify text using support vector machines - methods, theory and algorithms , 2002, The Kluwer international series in engineering and computer science.

[19]  David J. Crisp,et al.  Uniqueness of the SVM Solution , 1999, NIPS.

[20]  Michael I. Posner,et al.  Cognition (2nd ed.). , 1987 .

[21]  Mario Martín,et al.  On-Line Support Vector Machine Regression , 2002, ECML.