A Multi-class Incremental and Decremental SVM Approach Using Adaptive Directed Acyclic Graphs

Multi-class approaches for SVMs are based on composition of binary SVM classifiers. Due to the numerous binary classifiers to be considered, for large training sets, this approach is known to be time expensive. In our approach, we improve time efficiency using concurrently two strategies: incremental training and reduction of trained binary SVMs. We present the exact migration conditions for the binary SVMs during their incremental training. We rewrite these conditions for the case when the regularization parameter is optimized. The obtained results are applied to a multi-class incremental / decremental SVM based on the Adaptive Directed Acyclic Graph. The regularization parameter is optimized on-line, and not by retraining the SVM with all input samples for each value of the regularization parameter.

[1]  Stefan Rüping,et al.  Incremental Learning with Support Vector Machines , 2001, ICDM.

[2]  James Theiler,et al.  Accurate On-line Support Vector Regression , 2003, Neural Computation.

[3]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Razvan Andonie,et al.  Implementation Issues of an Incremental and Decremental SVM , 2008, ICANN.

[5]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[6]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[7]  Liu Zhibin,et al.  LATTICESVM — A new method for multi-class Support Vector machines , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  Klaus-Robert Müller,et al.  Incremental Support Vector Learning: Analysis, Implementation and Applications , 2006, J. Mach. Learn. Res..

[9]  Wenjian Wang,et al.  Online prediction model based on support vector machine , 2008, Neurocomputing.

[10]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[11]  Rui Wang,et al.  An Improvement of One-Against-One Method for Multi-Class Support Vector Machine , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[12]  Marimuthu Palaniswami,et al.  Incremental training of support vector machines , 2005, IEEE Transactions on Neural Networks.

[13]  Yann LeCun,et al.  Large Scale Online Learning , 2003, NIPS.

[14]  Gert Cauwenberghs,et al.  SVM incremental learning, adaptation and optimization , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[15]  Sean Luke,et al.  Evolving kernels for support vector machine classification , 2007, GECCO '07.

[16]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[17]  Mario Martín Muñoz.  On-line support vector machines for function approximation , 2002 .

[18]  Shigeo Abe.  Batch Support Vector Training Based on Exact Incremental Training , 2008, ICANN.

[19]  Boonserm Kijsirikul,et al.  Multiclass support vector machines using adaptive directed acyclic graph , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[20]  Ethem Alpaydin,et al.  Support Vector Machines for Multi-class Classification , 1999, IWANN.

[21]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[22]  David J. Crisp,et al.  Uniqueness of the SVM Solution , 1999, NIPS.

[23]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[24]  Liva Ralaivola,et al.  Incremental Learning Algorithms for Classification and Regression: local strategies , 2002 .

[25]  Glenn Fung,et al.  Incremental Support Vector Machine Classification , 2002, SDM.