Nonlinear Adaptive Filtering With Kernel Set-Membership Approach

This paper develops nonlinear kernel adaptive filtering algorithms based on the set-membership filtering (SMF) framework. The set-membership-based filtering approach is distinct from the conventional adaptive filtering approaches in that it aims for the filtering error being bounded in magnitude, as opposed to seeking to minimize the time average or ensemble average of the squared errors. The proposed kernel SMF algorithms feature selective updates of their parameter estimates by making discerning use of the input data, and selective increase of the dimension in the kernel expansion. These result in less computational cost and faster tracking without compromising the mean-squared error performance. We show, through convergence analysis, that the sequences of parameter estimates of our proposed algorithms are convergent, and the filtering error is asymptotically upper bounded in magnitude. Simulations are performed which show clearly the advantages of the proposed algorithms in terms of lower computational complexity, reduced dictionary size, and steady-state mean-squared errors comparable to existing algorithms.

[1]  Weifeng Liu,et al.  Kernel least mean square algorithm with constrained growth , 2009, Signal Process..

[2]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[3]  Shirish Nagaraj,et al.  Set-membership filtering and a set-membership normalized LMS algorithm with an adaptive step size , 1998, IEEE Signal Processing Letters.

[4]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[5]  Edward J. Powers,et al.  Optimal Volterra kernel estimation algorithms for a nonlinear communication system for PSK and QAM inputs , 2001, IEEE Trans. Signal Process..

[6]  Masahiro Yukawa,et al.  Adaptive Nonlinear Estimation Based on Parallel Projection Along Affine Subspaces in Reproducing Kernel Hilbert Space , 2015, IEEE Transactions on Signal Processing.

[7]  Yih-Fang Huang,et al.  Nonlinear Online Learning — A Kernel SMF Approach , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[8]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[9]  Rodrigo C. de Lamare,et al.  Set-membership adaptive kernel NLMS algorithms: Design and analysis , 2019, Signal Process..

[10]  Kutluyil Dogançay,et al.  Steady-state mean squared error and tracking performance analysis of the quasi-OBE algorithm , 2013, Signal Process..

[11]  Er-Wei Bai,et al.  Convergence Properties of the Membership Set , 1998, Autom..

[12]  Y. F. Huang,et al.  On the value of information in system identification - Bounded noise case , 1982, Autom..

[13]  Sergios Theodoridis,et al.  Online Distributed Learning Over Networks in RKH Spaces Using Random Fourier Features , 2017, IEEE Transactions on Signal Processing.

[14]  Masahiro Yukawa,et al.  Multikernel Adaptive Filtering , 2012, IEEE Transactions on Signal Processing.

[15]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[16]  Badong Chen,et al.  Learning Nonlinear Generative Models of Time Series With a Kalman Filter in RKHS , 2014, IEEE Transactions on Signal Processing.

[17]  Yih-Fang Huang,et al.  Distributed parameter estimation with selective cooperation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Yih-Fang Huang,et al.  BEACON: an adaptive set-membership filtering technique with sparse updates , 1999, IEEE Trans. Signal Process..

[19]  Sergios Theodoridis,et al.  Online Kernel-Based Classification Using Adaptive Projection Algorithms , 2008, IEEE Transactions on Signal Processing.

[20]  Jie Chen,et al.  Online Dictionary Learning for Kernel LMS , 2014, IEEE Transactions on Signal Processing.

[21]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[22]  Yih-Fang Huang,et al.  Kernelized set-membership approach to nonlinear adaptive filtering , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[23]  Yih-Fang Huang,et al.  Asymptotically convergent modified recursive least-squares with data-dependent updating and forgetting factor , 1985, 1985 24th IEEE Conference on Decision and Control.

[24]  Anke Schmeink,et al.  A Fast Converging Channel Estimation Algorithm for Wireless Sensor Networks , 2018, IEEE Transactions on Signal Processing.

[25]  Paulo Sergio Ramirez,et al.  Fundamentals of Adaptive Filtering , 2002 .

[26]  Yih-Fang Huang,et al.  Set-membership adaptive equalization and an updator-shared implementation for multiple channel communications systems , 1998, IEEE Trans. Signal Process..

[27]  S. Haykin,et al.  Kernel Least‐Mean‐Square Algorithm , 2010 .

[28]  Sergios Theodoridis,et al.  Sliding Window Generalized Kernel Affine Projection Algorithm Using Projection Mappings , 2008, EURASIP J. Adv. Signal Process..

[29]  Weifeng Liu,et al.  Kernel Adaptive Filtering , 2010 .

[30]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[31]  Narendra Ahuja,et al.  Online learning with kernels: Overcoming the growing sum problem , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[32]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[33]  Masahiro Yukawa,et al.  Efficient Dictionary-Refining Kernel Adaptive Filter With Fundamental Insights , 2016, IEEE Transactions on Signal Processing.

[34]  Paulo S. R. Diniz,et al.  On Data-Selective Adaptive Filtering , 2018, IEEE Transactions on Signal Processing.

[35]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[36]  Paul Honeine,et al.  Online Prediction of Time Series Data With Kernels , 2009, IEEE Transactions on Signal Processing.

[37]  Miguel Lázaro-Gredilla,et al.  Kernel Recursive Least-Squares Tracker for Time-Varying Regression , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Cédric Richard,et al.  Stochastic Behavior Analysis of the Gaussian Kernel Least-Mean-Square Algorithm , 2012, IEEE Transactions on Signal Processing.

[39]  Thomas Kailath,et al.  RKHS approach to detection and estimation problems-V: Parameter estimation , 1973, IEEE Trans. Inf. Theory.

[40]  Monson H. Hayes,et al.  Statistical Digital Signal Processing and Modeling , 1996 .

[41]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[42]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[43]  S. Kung Kernel Methods and Machine Learning , 2014 .

[44]  Badong Chen,et al.  A FIXED-BUDGET QUANTIZED KERNEL LEAST MEAN SQUARE ALGORITHM , 2012 .