Towards the online learning with Kernels in classification and regression

In this paper, optimization models and algorithms for online learning with Kernels (OLK) in classification and regression are proposed in a reproducing Kernel Hilbert Space (RKHS) by solving a constrained optimization model. The “Forgetting” factor in the model makes it possible that the memory requirement of the algorithm can be bounded as the learning process continues. The applications of the proposed OLK algorithms in classification and regression show their effectiveness in comparing with the state of art algorithms.

[1]  Guoqi Li,et al.  Identification of Wiener Systems With Clipped Observations , 2012, IEEE Trans. Signal Process..

[2]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[3]  Bernhard Schölkopf,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[4]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[5]  Yan Chen,et al.  Error tolerance based support vector machine for regression , 2011, Neurocomputing.

[6]  Zhengguo Li,et al.  Model-Based Online Learning With Kernels , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Guoqi Li,et al.  Online learning with kernels in classification and regression , 2012, 2012 IEEE Conference on Evolving and Adaptive Intelligent Systems.

[8]  Wei Xing Zheng,et al.  Identification of a Class of Nonlinear Autoregressive Models With Exogenous Inputs Based on Kernel Machines , 2011, IEEE Transactions on Signal Processing.

[9]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[10]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[11]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT.

[12]  Barbara Caputo,et al.  Bounded Kernel-Based Online Learning , 2009, J. Mach. Learn. Res..

[13]  Vojislav Kecman,et al.  Support vectors selection by linear programming , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[14]  Yoram Singer,et al.  The Forgetron: A Kernel-Based Perceptron on a Budget , 2008, SIAM J. Comput..