An Efficient Running Quantile Estimation Technique alongside Correntropy for Outlier Rejection in Online Regression

In this paper, online linear regression in the presence of non-Gaussian noise is addressed. In such environments, there are outliers in error samples (error between system output and labels) and/or the error does not follow a Gaussian distribution. Information theoretic measures such as error entropy criterion (EEC) and error correntropy criterion (ECC) are known for their superior performance compared to mean square error (MSE) in these cases. In this paper, an efficient technique of running quantile estimation based on quantization of error samples is introduced alongside which correntropy leads to lower steady state misalignment compared to previous algorithms.

[1]  Nitya Tiwari,et al.  A Technique with Low Memory and Computational Requirements for Dynamic Tracking of Quantiles , 2019, J. Signal Process. Syst..

[2]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[3]  P. P. Pokharel,et al.  Error Entropy, Correntropy and M-Estimation , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[4]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[5]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[6]  Sajjad Bahrami,et al.  Mitigating Outlier Effect in Online Regression: An Efficient Usage of Error Correntropy Criterion , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[7]  G. Blelloch.  Introduction to Data Compression * , 2022 .

[8]  Imrich Chlamtac,et al.  The P2 algorithm for dynamic calculation of quantiles and histograms without storing observations , 1985, CACM.

[9]  Zhiqiang He,et al.  Joint Channel Estimation and Impulsive Noise Mitigation in Underwater Acoustic OFDM Communication Systems , 2017, IEEE Transactions on Wireless Communications.

[10]  Ertem Tuncel,et al.  A New Approach To Online Regression Based On Maximum Correntropy Criterion , 2019, 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP).

[11]  José Carlos Príncipe,et al.  Kernel width adaptation in information theoretic cost functions , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Artur Gramacki,et al.  Nonparametric Kernel Density Estimation and Its Computational Aspects , 2017 .

[13]  Ghosheh Abed Hodtani,et al.  A New Information Theoretic Relation Between Minimum Error Entropy and Maximum Correntropy , 2018, IEEE Signal Processing Letters.

[14]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[15]  Sheng Zhang,et al.  Adaptive Filtering Under a Variable Kernel Width Maximum Correntropy Criterion , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.

[16]  Milica Stojanovic,et al.  Underwater acoustic communication channels: Propagation models and statistical characterization , 2009, IEEE Communications Magazine.

[17]  Zhiqiang He,et al.  Joint Channel and Impulsive Noise Estimation in Underwater Acoustic OFDM Systems , 2017, IEEE Transactions on Vehicular Technology.

[18]  Anis Yazidi,et al.  Smooth estimates of multiple quantiles in dynamically varying data streams , 2019, Pattern Analysis and Applications.

[19]  Ognjen Arandjelović,et al.  Targeted Adaptable Sample for Accurate and Efficient Quantile Estimation in Non-Stationary Data Streams , 2019, Mach. Learn. Knowl. Extr..

[20]  Sheng Zhang,et al.  Combined-Step-Size Affine Projection Sign Algorithm for Robust Adaptive Filtering in Impulsive Interference Environments , 2016, IEEE Transactions on Circuits and Systems II: Express Briefs.

[21]  Robert W. Newcomb,et al.  A Normalized Least Mean Squares Algorithm With a Step-Size Scaler Against Impulsive Measurement Noise , 2013, IEEE Transactions on Circuits and Systems II: Express Briefs.

[22]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[23]  Badong Chen,et al.  Kernel adaptive filtering with maximum correntropy criterion , 2011, The 2011 International Joint Conference on Neural Networks.

[24]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[25]  Haixin Sun,et al.  Impulsive Noise Mitigation in Underwater Acoustic OFDM Systems , 2016, IEEE Transactions on Vehicular Technology.