Quantized generalized maximum correntropy criterion based kernel recursive least squares for online time series prediction

Abstract With the rapid development of information theoretic learning, the maximum correntropy criterion (MCC) has been widely used in time series prediction area. Especially, the kernel recursive least squares (KRLS) based on MCC is studied recently due to its online recursive form and the ability to resist noise and be robust in non-Gaussian environments. However, it is not always an optimal choice that using the correntropy, which is calculated by default Gaussian kernel function, to describe the local similarity between variables. Besides, the computational burden of MCC based KRLS will raise as data size increases, thus causing difficulties in accommodating time-varying environments. Therefore, this paper proposes a quantized generalized MCC (QGMCC) to solve the above problem. Specifically, a generalized MCC (GMCC) is utilized to enhance the accuracy and flexibility in calculating the correntropy. In order to solve the problem of computational complexity, QGMCC quantizes the input space and upper bounds the network size by vector quantization (VQ) method. Furthermore, QGMCC is applied to KRLS and forming a computationally efficient and precisely predictive algorithm. After that, the improved algorithm named quantized kernel recursive generalized maximum correntropy (QKRGMC) is set up and the derivation process is also given. Experimental results of one benchmark dataset and two real-world datasets are present to verify the effectiveness of the online prediction algorithm.

[1]  Badong Chen,et al.  Kernel adaptive filtering with maximum correntropy criterion , 2011, The 2011 International Joint Conference on Neural Networks.

[2]  Jun Wang,et al.  Chaotic Time Series Prediction Based on a Novel Robust Echo State Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Zongze Wu,et al.  Convex regularized recursive maximum correntropy algorithm , 2016, Signal Process..

[4]  C. L. Philip Chen,et al.  Structured Manifold Broad Learning System: A Manifold Perspective for Large-Scale Chaotic Time Series Analysis and Prediction , 2019, IEEE Transactions on Knowledge and Data Engineering.

[5]  Hongbin Zhang,et al.  Kernel Recursive Generalized Maximum Correntropy , 2017, IEEE Signal Processing Letters.

[6]  Tie Qiu,et al.  Nonuniform State Space Reconstruction for Multivariate Chaotic Time Series , 2019, IEEE Transactions on Cybernetics.

[7]  Ali H. Sayed,et al.  Fundamentals Of Adaptive Filtering , 2003 .

[8]  Steven L. Scott,et al.  Predicting the Present with Bayesian Structural Time Series , 2013 .

[9]  Shukai Duan,et al.  Quantized kernel maximum correntropy and its mean square convergence analysis , 2017, Digit. Signal Process..

[10]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[11]  Wentao Ma,et al.  Robust kernel adaptive filters based on mean p-power error for noisy chaotic time series prediction , 2017, Eng. Appl. Artif. Intell..

[12]  Hazem Nounou,et al.  Multiscale Gaussian process regression-based generalized likelihood ratio test for fault detection in water distribution networks , 2019, Eng. Appl. Artif. Intell..

[13]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[14]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[15]  Nanning Zheng,et al.  Quantized Minimum Error Entropy Criterion , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Antonio Artés-Rodríguez,et al.  Information-Theoretic Linear Feature Extraction Based on Kernel Density Estimators: A Review , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[17]  Reza Mohammadi,et al.  A new hybrid evolutionary based RBF networks method for forecasting time series: A case study of forecasting emergency supply demand time series , 2014, Eng. Appl. Artif. Intell..

[18]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[19]  Deniz Erdoğmuş.  INFORMATION THEORETIC LEARNING: RENYI'S ENTROPY AND ITS APPLICATIONS TO ADAPTIVE SYSTEM TRAINING , 2002 .

[20]  Zongze Wu,et al.  Kernel recursive maximum correntropy , 2015, Signal Process..

[21]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[22]  Jian-Wu Xu,et al.  Nonlinear Signal Processing Based on Reproducing Kernel Hilbert Space , 2010 .

[23]  Xin Wang,et al.  Mixture correntropy for robust learning , 2018, Pattern Recognit..

[24]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[25]  Haibo He,et al.  SDE: A Novel Clustering Framework Based on Sparsity-Density Entropy , 2018, IEEE Transactions on Knowledge and Data Engineering.

[26]  Tie Qiu,et al.  Multivariate Chaotic Time Series Online Prediction Based on Improved Kernel Recursive Least Squares Algorithm , 2019, IEEE Transactions on Cybernetics.

[27]  Qing Song,et al.  A linear recurrent kernel online learning algorithm with sparse updates , 2014, Neural Networks.

[28]  Lu Xu,et al.  Robust Blind Learning Algorithm for Nonlinear Equalization Using Input Decision Information , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Md. Mustafizur Rahman,et al.  Layered Ensemble Architecture for Time Series Forecasting , 2016, IEEE Transactions on Cybernetics.

[30]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[31]  Hamoon Azizsoltani,et al.  Adaptive sequential strategy for risk estimation of engineering systems using Gaussian process regression active learning , 2018, Eng. Appl. Artif. Intell..

[32]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.

[33]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..