Kernel recurrent system trained by real-time recurrent learning algorithm

This paper presents a kernelized version of recurrent systems (KRS) and develops a kernel real-time recurrent learning (KRTRL) algorithm to train KRS. To avoid instabilities during training, the teacher forcing technique is adopted to modify the KRTRL learning. The proposed algorithms compared with the KLMS in Lorenz time series prediction. The prediction performances of the proposed algorithm outperform the KLMS significantly.

[1]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[2]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[3]  Badong Chen,et al.  A novel extended kernel recursive least squares algorithm , 2012, Neural Networks.

[4]  Ronald J. Williams,et al.  Experimental Analysis of the Real-time Recurrent Learning Algorithm , 1989 .

[5]  Weifeng Liu,et al.  The Kernel Least-Mean-Square Algorithm , 2008, IEEE Transactions on Signal Processing.

[6]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[7]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[8]  Weifeng Liu,et al.  Extended Kernel Recursive Least Squares Algorithm , 2009, IEEE Transactions on Signal Processing.

[9]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[10]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[11]  Gustavo Camps-Valls,et al.  Explicit recursivity into reproducing kernel Hilbert spaces , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  M. Sewell,et al.  Kernel Methods , 2017, Encyclopedia of Machine Learning and Data Mining.

[13]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[16]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[17]  Baver Okutmustur.  Reproducing kernel Hilbert spaces , 2005 .

[18]  Hao Shen,et al.  Fast Kernel-Based Independent Component Analysis , 2009, IEEE Transactions on Signal Processing.

[19]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .