Kernel least mean square algorithm with constrained growth

The linear least mean squares (LMS) algorithm has been recently extended to a reproducing kernel Hilbert space, resulting in an adaptive filter built from a weighted sum of kernel functions evaluated at each incoming data sample. With time, the size of the filter as well as the computation and memory requirements increase. In this paper, we shall propose a new efficient methodology for constraining the increase in length of a radial basis function (RBF) network resulting from the kernel LMS algorithm without significant sacrifice on performance. The method involves sequential Gaussian elimination steps on the Gram matrix to test the linear dependency of the feature vector corresponding to each new input with all the previous feature vectors. This gives an efficient method of continuing the learning as well as restricting the number of kernel functions used.

[1]  Saleem A. Kassam,et al.  Channel Equalization Using Adaptive Complex Radial Basis Function Networks , 1995, IEEE J. Sel. Areas Commun..

[2]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[3]  Ali H. Sayed,et al.  Fundamentals Of Adaptive Filtering , 2003 .

[4]  Weifeng Liu,et al.  Kernel Affine Projection Algorithms , 2008, EURASIP J. Adv. Signal Process..

[5]  Weifeng Liu,et al.  The Kernel Least-Mean-Square Algorithm , 2008, IEEE Transactions on Signal Processing.

[6]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[7]  André da Motta Salles Barreto,et al.  GOLS - Genetic orthogonal least squares algorithm for training RBF networks , 2006, Neurocomputing.

[8]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[9]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[10]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[11]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[12]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[13]  D. Erdogmus,et al.  Nonlinear channel equalization using multilayer perceptrons with information-theoretic criterion , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[14]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[15]  Weifeng Liu,et al.  Kernel LMS , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[16]  Ignacio Santamaría,et al.  Nonlinear System Identification using a New Sliding-Window Kernel RLS Algorithm , 2007, J. Commun..

[17]  齋藤 三郎,et al.  Theory of reproducing kernels and its applications , 1988 .

[18]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[19]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[20]  S. Haykin,et al.  Kernel Least‐Mean‐Square Algorithm , 2010 .

[21]  Saburou Saitoh,et al.  Theory of Reproducing Kernels and Its Applications , 1988 .

[22]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[23]  Henry Leung,et al.  Detection of small objects in clutter using a GA-RBF neural network , 2002 .

[24]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[25]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[26]  Gene H. Golub,et al.  Matrix computations , 1983 .

[27]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[28]  Bernie Mulgrew,et al.  IEEE Workshop on Neural Networks for Signal Processing , 1995 .

[29]  F. Takens.  Detecting strange attractors in turbulence , 1981 .