A Regularized Correntropy Framework for Robust Pattern Recognition

This letter proposes a new multiple linear regression model using regularized correntropy for robust pattern recognition. First, we motivate the use of correntropy to improve the robustness of the classical mean square error (MSE) criterion that is sensitive to outliers. Then an l1 regularization scheme is imposed on the correntropy to learn robust and sparse representations. Based on the half-quadratic optimization technique, we propose a novel algorithm to solve the nonlinear optimization problem. Second, we develop a new correntropy-based classifier based on the learned regularization scheme for robust object recognition. Extensive experiments over several applications confirm that the correntropy-based l1 regularization can improve recognition accuracy and receiver operator characteristic curves under noise corruption and occlusion.

[1]  B. V. Vijaya Kumar,et al.  Minimum-variance synthetic discriminant functions , 1986 .

[2]  Mohammed Bennamoun,et al.  Linear Regression for Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Yun-Hong Wang,et al.  A BEMD based muti-layer face matching: From near infrared to visual images , 2009, 2009 International Conference on Machine Learning and Cybernetics.

[5]  Wei-Shi Zheng,et al.  Principal Component Analysis Based on Nonparametric Maximum Entropy , 2010 .

[6]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Ran He,et al.  Robust Discriminant Analysis Based on Nonparametric Maximum Entropy , 2009, ACML.

[8]  HeRan,et al.  A regularized correntropy framework for robust pattern recognition , 2011 .

[9]  Mia Hubert,et al.  Fast and robust discriminant analysis , 2004, Comput. Stat. Data Anal..

[10]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[11]  D Casasent,et al.  Multivariant technique for multiclass pattern recognition. , 1980, Applied optics.

[12]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[13]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[14]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[15]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[16]  Paul A. Viola,et al.  Empirical Entropy Manipulation for Real-World Problems , 1995, NIPS.

[17]  Aleix M. Martinez,et al.  Support Vector Machines in face recognition with occlusions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Hong-Jie Xing,et al.  Training extreme learning machine via regularized correntropy criterion , 2012, Neural Computing and Applications.

[19]  Aleix M. Martínez,et al.  Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Aleix M. Martínez,et al.  Face recognition with occlusions in the training and testing sets , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[21]  C. Croux,et al.  Robust linear discriminant analysis using S‐estimators , 2001 .

[22]  Kari Torkkola,et al.  Feature Extraction by Non-Parametric Mutual Information Maximization , 2003, J. Mach. Learn. Res..

[23]  Charles V. Stewart,et al.  Robust Computer Vision: An Interdisciplinary Challenge , 2000, Comput. Vis. Image Underst..

[24]  John W. Fisher,et al.  Learning from Examples with Information Theoretic Criteria , 2000, J. VLSI Signal Process..

[25]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[26]  A. Martínez,et al.  The AR face databasae , 1998 .

[27]  Weifeng Liu,et al.  A low complexity robust detector in impulsive noise , 2009, Signal Process..

[28]  E. Candes,et al.  11-magic : Recovery of sparse signals via convex programming , 2005 .

[29]  Stan Z. Li,et al.  Face recognition based on nearest linear combinations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[30]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[31]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[32]  Stan Z. Li,et al.  Face recognition using the nearest feature line method , 1999, IEEE Trans. Neural Networks.

[33]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[34]  Jen-Tzung Chien,et al.  Discriminant Waveletfaces and Nearest Feature Classifiers for Face Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Marios Savvides,et al.  Correlation Pattern Recognition for Face Recognition , 2006, Proceedings of the IEEE.

[37]  Shaohua Kevin Zhou,et al.  Variational Graph Embedding for Globally and Locally Consistent Feature Extraction , 2009, ECML/PKDD.

[38]  Ran He,et al.  Principal component analysis based on non-parametric maximum entropy , 2010, Neurocomputing.

[39]  Katsushi Ikeuchi,et al.  Detectability, Uniqueness, and Reliability of Eigen Windows for Stable Verification of Partially Occluded Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[41]  Horst Bischof,et al.  Robust Recognition Using Eigenimages , 2000, Comput. Vis. Image Underst..

[42]  Na Liu,et al.  A facial sparse descriptor for single image based face recognition , 2012, Neurocomputing.

[43]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[44]  Dong Yi,et al.  Face Matching Between Near Infrared and Visible Light Images , 2007, ICB.

[45]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[46]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[47]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[48]  Jian Yang,et al.  Recursive robust least squares support vector regression based on maximum correntropy criterion , 2012, Neurocomputing.

[49]  Ran He,et al.  Nearest Feature Line: A Tangent Approximation , 2008, 2008 Chinese Conference on Pattern Recognition.

[50]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[52]  José Carlos Príncipe,et al.  Generalized correlation function: definition, properties, and application to blind equalization , 2006, IEEE Transactions on Signal Processing.

[53]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[54]  W. Fung,et al.  High Breakdown Estimation for Multiple Populations with Applications to Discriminant Analysis , 2000 .

[55]  Yoshua Bengio,et al.  Entropy Regularization , 2006, Semi-Supervised Learning.

[56]  José Carlos Príncipe,et al.  Enhancing the correntropy MACE filter with random projections , 2008, Neurocomputing.

[57]  Jian Yang,et al.  Regularized Robust Coding for Face Recognition , 2012, IEEE Transactions on Image Processing.

[58]  Deniz Erdogmus,et al.  Information Theoretic Learning , 2005, Encyclopedia of Artificial Intelligence.

[59]  David R. Musicant,et al.  Robust Linear and Support Vector Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Sanja Fidler,et al.  Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Weifeng Liu,et al.  Correntropy: A Localized Similarity Measure , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[62]  Ran He,et al.  Two-Stage Nonnegative Sparse Representation for Large-Scale Face Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[63]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[64]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[65]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[67]  J. Príncipe,et al.  Energy, entropy and information potential for neural computation , 1998 .

[68]  Douglas M. Hawkins,et al.  High-Breakdown Linear Discriminant Analysis , 1997 .

[69]  Weifeng Liu,et al.  The correntropy MACE filter , 2009, Pattern Recognit..

[70]  Bao-Gang Hu,et al.  Robust feature extraction via information theoretic learning , 2009, ICML '09.

[71]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[72]  D. Casasent,et al.  Minimum average correlation energy filters. , 1987, Applied optics.

[73]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[74]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[75]  B V Kumar,et al.  Tutorial survey of composite filter designs for optical correlators. , 1992, Applied optics.

[76]  Thomas S. Huang,et al.  Joint dynamic sparse representation for multi-view face recognition , 2012, Pattern Recognit..