Maximum Correntropy Criterion-Based Sparse Subspace Learning for Unsupervised Feature Selection

High-dimensional data contain not only redundancy but also noises produced by the sensors. These noises are usually non-Gaussian distributed. The metrics based on Euclidean distance are not suitable for these situations in general. In order to select the useful features and combat the adverse effects of the noises simultaneously, a robust sparse subspace learning method in unsupervised scenario is proposed in this paper based on the maximum correntropy criterion that shows strong robustness against outliers. Furthermore, an iterative strategy based on half quadratic and an accelerated block coordinate update is proposed. The convergence analysis of the proposed method is also carried out to ensure the convergence to a reliable solution. Extensive experiments are conducted on real-world data sets to show that the new method can filter out the outliers and outperform several state-of-the-art unsupervised feature selection methods.

[1]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[2]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[3]  Wotao Yin,et al.  A Globally Convergent Algorithm for Nonconvex Optimization Based on Block Coordinate Update , 2014, J. Sci. Comput..

[4]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[5]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[6]  Yangyang Xu,et al.  Alternating proximal gradient method for sparse nonnegative Tucker decomposition , 2013, Mathematical Programming Computation.

[7]  Jiawei Han,et al.  Joint Feature Selection and Subspace Learning , 2011, IJCAI.

[8]  Wotao Yin,et al.  A Primer on Coordinate Descent Algorithms , 2016, 1610.00040.

[9]  Tieniu Tan,et al.  l2, 1 Regularized correntropy for robust feature selection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Yangyang Xu,et al.  On the convergence of higher-order orthogonal iteration , 2015 .

[11]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[12]  Zongze Wu,et al.  Robust Hammerstein Adaptive Filtering under Maximum Correntropy Criterion , 2015, Entropy.

[13]  Xi Liu,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[14]  Ming Yan,et al.  Coordinate Friendly Structures, Algorithms and Applications , 2016, ArXiv.

[15]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[16]  Donald Geman,et al.  Constrained Restoration and the Recovery of Discontinuities , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Hong Cheng,et al.  A robust elastic net approach for feature learning , 2014, J. Vis. Commun. Image Represent..

[18]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[19]  Nanning Zheng,et al.  Generalized Correntropy for Robust Adaptive Filtering , 2015, IEEE Transactions on Signal Processing.

[20]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[21]  Lei Wang,et al.  Global and Local Structure Preservation for Feature Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[23]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[24]  Yang Wang,et al.  Mutual information-based method for selecting informative feature sets , 2013, Pattern Recognit..

[25]  Nanning Zheng,et al.  Steady-State Mean-Square Error Analysis for Adaptive Filtering under the Maximum Correntropy Criterion , 2014, IEEE Signal Processing Letters.

[26]  Nanning Zheng,et al.  Convergence of a Fixed-Point Algorithm under Maximum Correntropy Criterion , 2015, IEEE Signal Processing Letters.

[27]  Simon C. K. Shiu,et al.  Unsupervised feature selection by regularized self-representation , 2015, Pattern Recognit..

[28]  Xiaolei Wang,et al.  Non-negative matrix factorization by maximizing correntropy for cancer clustering , 2013, BMC Bioinformatics.

[29]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[30]  Witold Pedrycz,et al.  Discriminative sparse subspace learning and its application to unsupervised feature selection. , 2016, ISA transactions.

[31]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Witold Pedrycz,et al.  Global and local structure preserving sparse subspace learning: An iterative approach to unsupervised feature selection , 2015, Pattern Recognit..

[33]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[34]  Yangyang Xu,et al.  Fast algorithms for Higher-order Singular Value Decomposition from incomplete data , 2014 .

[35]  Ran He,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Adrian S. Lewis,et al.  THEINEQUALITY FOR NONSMOOTH SUBANALYTIC FUNCTIONS WITH APPLICATIONS TO , 2007 .

[37]  Witold Pedrycz,et al.  Subspace learning for unsupervised feature selection via matrix factorization , 2015, Pattern Recognit..

[38]  Jose C. Principe,et al.  Information Theoretic Learning - Renyi's Entropy and Kernel Perspectives , 2010, Information Theoretic Learning.

[39]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Badong Chen,et al.  Maximum Correntropy Estimation Is a Smoothed MAP Estimation , 2012, IEEE Signal Processing Letters.