Quantized kernel maximum correntropy and its mean square convergence analysis

Online vector quantization (VQ) method has been successfully applied to the kernel adaptive filters (KAFs) for curbing their linearly growing radial basis function (RBF) network, thereby generating a family of quantized KAFs (QKAFs). However, the most existing QKAFs are based on the mean square error (MSE) criterion, which is actually not a good choice for non-Gaussian signals. In this paper, a new quantized kernel adaptive filter called quantized kernel maximum correntropy (QKMC) is developed, which is robust to large outliers or impulsive noises. The mean square convergence analysis for QKMC is conducted, and a sufficient condition for guaranteeing convergence is therefore obtained. The filtering accuracy of QKMC is also proved to be higher than that of the representative quantized kernel least mean square (QKLMS). In addition, to make full use of the information hidden in the input and the output spaces, we further propose a modified QKMC based on bilateral gradient (QKMC-BG). To limit the final network size of QKMC-BG, the QKMC-BG with fixed budget (QKMC-BG-FB) is also developed. Simulation results under the cases of Gaussian and non-Gaussian noises are presented to validate the proposed QKMC, QKMC-BG and QKMC-BG-FB. We propose a new QKMC based on maximum correntropy criterion.The mean square convergence analysis for QKMC is performed.The QKMC-BG is proposed by using bilateral gradient.The fixed budget version of QKMC-BG is also developed.Gaussian and non-Gaussian noises are considered to validate the proposed methods.

[1]  Zongze Wu,et al.  Kernel recursive maximum correntropy , 2015, Signal Process..

[2]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[3]  Weifeng Liu,et al.  Kernel Adaptive Filtering: A Comprehensive Introduction , 2010 .

[4]  Nanning Zheng,et al.  Kernel minimum error entropy algorithm , 2013, Neurocomputing.

[5]  Ran He,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Qing Song,et al.  A linear recurrent kernel online learning algorithm with sparse updates , 2014, Neural Networks.

[7]  Badong Chen,et al.  Maximum Correntropy Estimation Is a Smoothed MAP Estimation , 2012, IEEE Signal Processing Letters.

[8]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[9]  Weifeng Liu,et al.  Nonlinear Adaptive Filtering in Kernel Spaces , 2014 .

[10]  Badong Chen,et al.  A FIXED-BUDGET QUANTIZED KERNEL LEAST MEAN SQUARE ALGORITHM , 2012 .

[11]  Binwei Weng,et al.  Nonlinear system identification in impulsive environments , 2005, IEEE Transactions on Signal Processing.

[12]  Nanning Zheng,et al.  Generalized Correntropy for Robust Adaptive Filtering , 2015, IEEE Transactions on Signal Processing.

[13]  Zongze Wu,et al.  Robust Hammerstein Adaptive Filtering under Maximum Correntropy Criterion , 2015, Entropy.

[14]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[15]  Jiandong Duan,et al.  Diffusion maximum correntropy criterion algorithms for robust distributed estimation , 2015, Digit. Signal Process..

[16]  Zongze Wu,et al.  Minimum Error Entropy Algorithms with Sparsity Penalty Constraints , 2015, Entropy.

[17]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.

[18]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[19]  Badong Chen,et al.  A dictionary updating scheme incorporating words elimination into Quantized Kernel Least-Mean-Squares for changing environments , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[20]  Weifeng Liu,et al.  An Information Theoretic Approach of Designing Sparse Kernel Adaptive Filters , 2009, IEEE Transactions on Neural Networks.

[21]  Weifeng Liu,et al.  The Kernel Least-Mean-Square Algorithm , 2008, IEEE Transactions on Signal Processing.

[22]  Weifeng Liu,et al.  Kernel Affine Projection Algorithms , 2008, EURASIP J. Adv. Signal Process..

[23]  Badong Chen,et al.  Quantized Kernel Least Mean Square Algorithm , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Peng Wang,et al.  A robust DOA estimator based on the correntropy in alpha-stable noise environments , 2017, Digit. Signal Process..

[25]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[26]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[27]  José Carlos Príncipe,et al.  Quantized mixture kernel least mean square , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[28]  C. K. Michael Tse,et al.  A modified quantized kernel least mean square algorithm for prediction of chaotic time series , 2016, Digit. Signal Process..

[29]  Badong Chen,et al.  Quantized Kernel Recursive Least Squares Algorithm , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[31]  V. Miranda,et al.  Entropy and Correntropy Against Minimum Square Error in Offline and Online Three-Day Ahead Wind Power Forecasting , 2009, IEEE Transactions on Power Systems.

[32]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[33]  Badong Chen,et al.  Kernel adaptive filtering with maximum correntropy criterion , 2011, The 2011 International Joint Conference on Neural Networks.

[34]  Ran He,et al.  Robust Principal Component Analysis Based on Maximum Correntropy Criterion , 2011, IEEE Transactions on Image Processing.

[35]  Tianshuang Qiu,et al.  A robust correntropy based subspace tracking algorithm in impulsive noise environments , 2017, Digit. Signal Process..

[36]  Nanning Zheng,et al.  Steady-State Mean-Square Error Analysis for Adaptive Filtering under the Maximum Correntropy Criterion , 2014, IEEE Signal Processing Letters.

[37]  Shukai Duan,et al.  Simplified quantised kernel least mean square algorithm with fixed budget , 2016 .

[38]  Badong Chen,et al.  Quantised kernel least mean square with desired signal smoothing , 2015 .

[39]  Liming Shi,et al.  Convex Combination of Adaptive Filters under the Maximum Correntropy Criterion in Impulsive Interference , 2014, IEEE Signal Processing Letters.

[40]  Wentao Ma,et al.  Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments , 2015, J. Frankl. Inst..

[41]  José Carlos Príncipe,et al.  Using Correntropy as a cost function in linear adaptive filters , 2009, 2009 International Joint Conference on Neural Networks.

[42]  Badong Chen,et al.  Adaptive recursive algorithm with logarithmic transformation for nonlinear system identification in α-stable noise , 2015, Digit. Signal Process..

[43]  Hai Quang Dam,et al.  Deleted DOI: Postfiltering Using Multichannel Spectral Estimation in Multispeaker Environments , 2008 .