Robust Learning With Kernel Mean $p$ -Power Error Loss

Correntropy is a second order statistical measure in kernel space, which has been successfully applied in robust learning and signal processing. In this paper, we define a nonsecond order statistical measure in kernel space, called the kernel mean- ${p}$ power error (KMPE), including the correntropic loss (C-Loss) as a special case. Some basic properties of KMPE are presented. In particular, we apply the KMPE to extreme learning machine (ELM) and principal component analysis (PCA), and develop two robust learning algorithms, namely ELM-KMPE and PCA-KMPE. Experimental results on synthetic and benchmark data show that the developed algorithms can achieve better performance when compared with some existing methods.

[1]  Hong-Jie Xing,et al.  Training extreme learning machine via regularized correntropy criterion , 2012, Neural Computing and Applications.

[2]  Xizhao Wang,et al.  International journal of machine learning and cybernetics , 2010, Int. J. Mach. Learn. Cybern..

[3]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[4]  J. Tukey,et al.  The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data , 1974 .

[5]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[6]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Ian R. Petersen,et al.  Robustness and risk-sensitive filtering , 2002, IEEE Trans. Autom. Control..

[8]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[9]  Xi Liu,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[10]  Yu-Chiang Frank Wang,et al.  Undersampled Face Recognition via Robust Auxiliary Dictionary Learning , 2015, IEEE Transactions on Image Processing.

[11]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[12]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[13]  Jianzhong Wang,et al.  Excavation Equipment Recognition Based on Novel Acoustic Statistical Features , 2017, IEEE Transactions on Cybernetics.

[14]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Yaonan Wang,et al.  Data Partition Learning With Multiple Extreme Learning Machines , 2015, IEEE Transactions on Cybernetics.

[16]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[17]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[18]  Chris H. Q. Ding,et al.  K-means clustering via principal component analysis , 2004, ICML.

[19]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[20]  José Carlos Príncipe,et al.  The C-loss function for pattern classification , 2014, Pattern Recognit..

[21]  Nanning Zheng,et al.  Smoothed least mean p-power error criterion for adaptive filtering , 2015, Digit. Signal Process..

[22]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[23]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[24]  Nojun Kwak,et al.  Generalized mean for robust principal component analysis , 2016, Pattern Recognit..

[25]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[26]  Shang-Hong Lai Robust Image Matching under Partial Occlusion and Spatially Varying Illumination Change , 2000, Comput. Vis. Image Underst..

[27]  Yan Yang,et al.  Dimension Reduction With Extreme Learning Machine , 2016, IEEE Transactions on Image Processing.

[28]  Chien-Cheng Tseng,et al.  Least mean p-power error criterion for adaptive FIR filter , 1994, IEEE J. Sel. Areas Commun..

[29]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Nanning Zheng,et al.  Convergence of a Fixed-Point Algorithm under Maximum Correntropy Criterion , 2015, IEEE Signal Processing Letters.

[31]  Nanning Zheng,et al.  Steady-State Mean-Square Error Analysis for Adaptive Filtering under the Maximum Correntropy Criterion , 2014, IEEE Signal Processing Letters.

[32]  Jason Jianjun Gu,et al.  An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[33]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[34]  Badong Chen,et al.  Efficient and robust deep learning with Correntropy-induced loss function , 2015, Neural Computing and Applications.

[35]  Yimin Yang,et al.  Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning , 2016, IEEE Transactions on Cybernetics.

[36]  Nanning Zheng,et al.  Correntropy Maximization via ADMM: Application to Robust Hyperspectral Unmixing , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  Nanning Zheng,et al.  Generalized Correntropy for Robust Adaptive Filtering , 2015, IEEE Transactions on Signal Processing.

[39]  Carla E. Brodley,et al.  Proceedings of the twenty-first international conference on Machine learning , 2004, International Conference on Machine Learning.

[40]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[41]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[42]  Ran He,et al.  Robust Principal Component Analysis Based on Maximum Correntropy Criterion , 2011, IEEE Transactions on Image Processing.

[43]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  I. Jolliffe Principal Component Analysis , 2002 .

[45]  Badong Chen,et al.  System Parameter Identification: Information Criteria and Algorithms , 2013 .

[46]  Peter J. Huber,et al.  Wiley Series in Probability and Mathematics Statistics , 2005 .

[47]  Yuan-Hai Shao,et al.  Principal Component Analysis Based on T𝓁1-norm Maximization , 2020, ArXiv.

[48]  Tieniu Tan,et al.  Robust Recovery of Corrupted Low-RankMatrix by Implicit Regularizers , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Ran He,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Constantine Kotropoulos,et al.  Robust Multidimensional Scaling Using a Maximum Correntropy Criterion , 2017, IEEE Transactions on Signal Processing.