Online learning based on iterative projections in sum space of linear and Gaussian reproducing kernel Hilbert spaces

We propose a novel multikernel adaptive filtering algorithm based on the iterative projections in the sum space of reproducing kernel Hilbert spaces. We employ linear and Gaussian kernels, envisioning an application to partially-linear-system identification/estimation. The algorithm is derived by reformulating the hyperplane projection along affine subspace (HYPASS) algorithm in the sum space. The projection is computable by virtue of Minh's theorem proved in 2010 as long as the input space has nonempty interior. Numerical examples show the efficacy of the proposed algorithm.

[1]  Di-Rong Chen,et al.  Partially-Linear Least-Squares Regularized Regression for System Identification , 2009, IEEE Transactions on Automatic Control.

[2]  Sun-Yuan Kung,et al.  Multikernel Least Mean Square Algorithm , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Badong Chen,et al.  Online efficient learning with quantized KLMS and L1 regularization , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[4]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[5]  Weifeng Liu,et al.  Kernel Adaptive Filtering , 2010 .

[6]  Wolfgang Härdle,et al.  Partially Linear Models , 2000 .

[7]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[8]  Masahiro Yukawa,et al.  An efficient kernel adaptive filtering algorithm using hyperplane projection along affine subspace , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[9]  Paulo Sergio Ramirez,et al.  Fundamentals of Adaptive Filtering , 2002 .

[10]  Masahiro Yukawa.  Nonlinear adaptive filtering techniques with multiple kernels , 2011, 2011 19th European Signal Processing Conference.

[11]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[12]  Masahiro Yukawa,et al.  Adaptive Learning in Cartesian Product of Reproducing Kernel Hilbert Spaces , 2014, IEEE Transactions on Signal Processing.

[13]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[14]  Miguel Lázaro-Gredilla,et al.  Kernel Recursive Least-Squares Tracker for Time-Varying Regression , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Steven C. H. Hoi,et al.  Fast Bounded Online Gradient Descent Algorithms for Scalable Kernel-Based Online Learning , 2012, ICML.

[16]  Claudio Gentile,et al.  Tracking the best hyperplane with a simple budget Perceptron , 2006, Machine Learning.

[17]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[18]  H. Minh,et al.  Some Properties of Gaussian Reproducing Kernel Hilbert Spaces and Their Implications for Function Approximation and Learning Theory , 2010 .

[19]  Masahiro Yukawa,et al.  Online model selection and learning by multikernel adaptive filtering , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[20]  José Carlos Príncipe,et al.  Mixture kernel least mean square , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[21]  Masahiro Yukawa,et al.  An efficient sparse kernel adaptive filtering algorithm based on isomorphism between functional subspace and Euclidean space , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Sergios Theodoridis,et al.  Online Kernel-Based Classification Using Adaptive Projection Algorithms , 2008, IEEE Transactions on Signal Processing.

[23]  Marc Moonen,et al.  Nonlinear Acoustic Echo Cancellation Based on a Sliding-Window Leaky Kernel Affine Projection Algorithm , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[24]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[25]  Masahiro Yukawa,et al.  Multikernel Adaptive Filtering , 2012, IEEE Transactions on Signal Processing.

[26]  Johan A. K. Suykens,et al.  Kernel based partially linear models and nonlinear identification , 2005, IEEE Transactions on Automatic Control.

[27]  Barbara Caputo,et al.  The projectron: a bounded kernel-based Perceptron , 2008, ICML '08.

[28]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[29]  Jie Chen,et al.  Online Dictionary Learning for Kernel LMS , 2014, IEEE Transactions on Signal Processing.

[30]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[31]  Weifeng Liu,et al.  Kernel Affine Projection Algorithms , 2008, EURASIP J. Adv. Signal Process..

[32]  Cédric Richard,et al.  Convex combinations of kernel adaptive filters , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[33]  Paul Honeine,et al.  Online Prediction of Time Series Data With Kernels , 2009, IEEE Transactions on Signal Processing.

[34]  Yih-Fang Huang,et al.  Kernelized set-membership approach to nonlinear adaptive filtering , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[35]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.