New Incremental Learning Algorithm With Support Vector Machines

Incremental learning is one of the most effective methods of learning accumulated data and large-scale data. The newly increased samples of the previously known works on incremental learning are usually independent and identically distributed. To study how dependent sampling methods influence the learning ability of incremental support vector machines (ISVM) algorithm, in this paper we introduce an ISVM based on Markov resampling (MR-ISVM), and give the experimental research on the learning ability of the MR-ISVM algorithm. The experimental results indicate that the MR-ISVM algorithm has not only smaller misclassification rates and sparser of the obtained classifiers, but also less total time of sampling and training compared to ISVM based on randomly independent sampling. We also compare it with other ISVM algorithms.

[1]  Jie Xu,et al.  The Generalization Ability of Online SVM Classification Based on Markov Sampling , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Ding-Xuan Zhou,et al.  Capacity of reproducing kernel spaces in learning theory , 2003, IEEE Transactions on Information Theory.

[3]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[4]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[5]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[6]  S. Smale,et al.  ONLINE LEARNING WITH MARKOV SAMPLING , 2009 .

[7]  Cunhe Li,et al.  The incremental learning algorithm with support vector machine based on hyperplane-distance , 2011, Applied Intelligence.

[8]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[9]  Yiming Ying,et al.  Support Vector Machine Soft Margin Classifiers: Error Analysis , 2004, J. Mach. Learn. Res..

[10]  Don R. Hush,et al.  Learning from dependent observations , 2007, J. Multivar. Anal..

[11]  Ingo Steinwart,et al.  Fast Rates for Support Vector Machines , 2005, COLT.

[12]  Stefan Rüping,et al.  Incremental Learning with Support Vector Machines , 2001, ICDM.

[13]  Mathukumalli Vidyasagar,et al.  Learning and Generalization: With Applications to Neural Networks , 2002 .

[14]  Ingo Steinwart,et al.  Consistency of support vector machines and other regularized kernel classifiers , 2005, IEEE Transactions on Information Theory.

[15]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[16]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[17]  Haofeng Zhang,et al.  A Fast Incremental Learning Algorithm Based on Twin Support Vector Machine , 2014, 2014 Seventh International Symposium on Computational Intelligence and Design.

[18]  W. Marsden.  I and J , 2012 .

[19]  Piyabute Fuangkhon,et al.  An Outpost Vector placement evaluation of an incremental learning algorithm for Support Vector Machine , 2011, The 2011 International Joint Conference on Neural Networks.

[20]  Andreas Christmann,et al.  Fast Learning from Non-i.i.d. Observations , 2009, NIPS.

[21]  Zongben Xu,et al.  The generalization performance of ERM algorithm with strongly mixing observations , 2009, Machine Learning.

[22]  Paul-Marie Samson,et al.  Concentration of measure inequalities for Markov chains and $\Phi$-mixing processes , 2000 .

[23]  Yiming Ying,et al.  Online Regularized Classification Algorithms , 2006, IEEE Transactions on Information Theory.

[24]  Yuan Yan Tang,et al.  Generalization Performance of Fisher Linear Discriminant Based on Markov Sampling , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Jie Xu,et al.  The Generalization Ability of SVM Classification Based on Markov Sampling , 2015, IEEE Transactions on Cybernetics.

[26]  Jie Xu,et al.  Generalization performance of Gaussian kernels SVMC based on Markov sampling , 2014, Neural Networks.

[27]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .