Explicit Recursive and Adaptive Filtering in Reproducing Kernel Hilbert Spaces

This brief presents a methodology to develop recursive filters in reproducing kernel Hilbert spaces. Unlike previous approaches that exploit the kernel trick on filtered and then mapped samples, we explicitly define the model recursivity in the Hilbert space. For that, we exploit some properties of functional analysis and recursive computation of dot products without the need of preimaging or a training dataset. We illustrate the feasibility of the methodology in the particular case of the γ-filter, which is an infinite impulse response filter with controlled stability and memory depth. Different algorithmic formulations emerge from the signal model. Experiments in chaotic and electroencephalographic time series prediction, complex nonlinear system identification, and adaptive antenna array processing demonstrate the potential of the approach for scenarios where recursivity and nonlinearity have to be readily combined.

[1]  Weifeng Liu,et al.  Fixed-budget kernel recursive least-squares , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Georg Dorffner,et al.  Neural Networks for Time Series Processing , 1996 .

[3]  Gustavo Camps-Valls,et al.  Explicit recursivity into reproducing kernel Hilbert spaces , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[5]  Weifeng Liu,et al.  The Kernel Least-Mean-Square Algorithm , 2008, IEEE Transactions on Signal Processing.

[6]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[7]  Ignacio Santamaría,et al.  A Sliding-Window Kernel RLS Algorithm and Its Application to Nonlinear Channel Identification , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[8]  Jesús Requena-Carrión,et al.  Learning non-linear time-scales with kernel gamma-filters , 2009, Neurocomputing.

[9]  Liva Ralaivola,et al.  Dynamical Modeling with Kernels for Nonlinear Time Series Prediction , 2003, NIPS.

[10]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[11]  J. Suykens,et al.  Recurrent least squares support vector machines , 2000 .

[12]  R. Harrison,et al.  Support Vector Machines for System Identification , 1998 .

[13]  Bernhard Schölkopf,et al.  A Unifying View of Wiener and Volterra Theory and Polynomial Kernel Regression , 2006, Neural Computation.

[14]  David M. Pozar,et al.  A relation between the active input impedance and the active element pattern of a phased array , 2003 .

[15]  Xiaohong Jiang,et al.  Generalized Two-Hop Relay for Flexible Delay Control in MANETs , 2012, IEEE/ACM Transactions on Networking.

[16]  Sergios Theodoridis,et al.  Online Kernel-Based Classification Using Adaptive Projection Algorithms , 2008, IEEE Transactions on Signal Processing.

[17]  José Luis Rojo-Álvarez,et al.  Support Vector Machines for Nonlinear Kernel ARMA System Identification , 2006, IEEE Transactions on Neural Networks.

[18]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[19]  Johan A. K. Suykens,et al.  Support Vector Machines: A Nonlinear Modelling and Control Perspective , 2001, Eur. J. Control.

[20]  Alex Simpkins,et al.  System Identification: Theory for the User, 2nd Edition (Ljung, L.; 1999) [On the Shelf] , 2012, IEEE Robotics & Automation Magazine.

[21]  Weifeng Liu,et al.  An Information Theoretic Approach of Designing Sparse Kernel Adaptive Filters , 2009, IEEE Transactions on Neural Networks.

[22]  A. Gretton,et al.  Support vector regression for black-box system identification , 2001, Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing (Cat. No.01TH8563).

[23]  M. Niranjan,et al.  Sequential support vector machines , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[24]  Giorgio Biagetti,et al.  Nonlinear System Identification: An Effective Framework Based on the Karhunen–LoÈve Transform , 2009, IEEE Transactions on Signal Processing.

[25]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[26]  L. Ralaivola,et al.  Time series filtering, smoothing and learning using the kernel Kalman filter , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[27]  J. Suykens,et al.  Subspace identification of Hammerstein systems using least squares support vector machines , 2005 .

[28]  Davide Mattera,et al.  Support Vector Machines for Signal Processing , 2005 .

[29]  Johan A. K. Suykens,et al.  Optimal control by least squares support vector machines , 2001, Neural Networks.

[30]  R. Schaback,et al.  Recursive Kernels , 2009 .

[31]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[32]  José Carlos Príncipe,et al.  The gamma-filter-a new class of adaptive IIR filters with restricted feedback , 1993, IEEE Trans. Signal Process..