Joint correntropy metric weighting and block diagonal regularizer for robust multiple kernel subspace clustering

Abstract Nonlinear kernel-based subspace clustering methods that can reveal the multi-cluster nonlinear structure of samples are an emerging research topic. However, the existing kernel subspace clustering methods have the following three flaws: 1) their clustering performance is largely determined by the chosen kernel function; 2) they may lack robustness in the presence of non-Gaussian noise and impulsive noise; and 3) their learned affinity matrix can not hold the desired block diagonal property for clustering purpose, which possibly leads to incorrect clustering when using spectral clustering. In this paper, we propose a Joint Robust Multiple Kernel Subspace Clustering (JMKSC) method for data clustering, which has two primary innovations. First, our multiple kernel weighting strategy introduces the correntropy metric weighting instead of a fixed, or inappropriately assigned weighting, which is more robust to the non-Gaussian noise and contributes to learning the optimal consensus kernel. Second, our method encourages acquiring an affinity matrix with the optimal block diagonal property based on the block diagonal regularizer (BDR) and the self-expressiveness property. Experiments on several different types of datasets confirm that the proposed JMKSC significantly outperforms several state-of-the-art single kernel and multiple kernel subspace clustering methods in terms of accuracy, NMI and purity.

[1]  Zhaohong Deng,et al.  A survey on soft subspace clustering , 2014, Inf. Sci..

[2]  Jiye Liang,et al.  A new distance with derivative information for functional k-means clustering algorithm , 2018, Inf. Sci..

[3]  LinLin Shen,et al.  Joint regularized nearest points for image set based face recognition , 2017, Image Vis. Comput..

[4]  Zhenhua Guo,et al.  Self-learning for face clustering , 2018, Pattern Recognit..

[5]  Tao Mei,et al.  Subspace Clustering by Block Diagonal Representation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  René Vidal,et al.  Algebraic Clustering of Affine Subspaces , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Tieniu Tan,et al.  l2, 1 Regularized correntropy for robust feature selection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Deyu Meng,et al.  Robust subspace clustering via penalized mixture of Gaussians , 2018, Neurocomputing.

[9]  Huan Xu,et al.  Provable Subspace Clustering: When LRR Meets SSC , 2013, IEEE Transactions on Information Theory.

[10]  Fanyu Bu A High-Order Clustering Algorithm Based on Dropout Deep Learning for Heterogeneous Data in Cyber-Physical-Social Systems , 2018, IEEE Access.

[11]  Zenglin Xu,et al.  Low-rank kernel learning for graph-based clustering , 2019, Knowl. Based Syst..

[12]  Rosane Minghim,et al.  Interactive Document Clustering Revisited: A Visual Analytics Approach , 2018, IUI.

[13]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[14]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[15]  Zhao Kang,et al.  Robust Graph Regularized Nonnegative Matrix Factorization for Clustering , 2017, ACM Trans. Knowl. Discov. Data.

[16]  Weifeng Liu,et al.  A low complexity robust detector in impulsive noise , 2009, Signal Process..

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  Bo Du,et al.  Robust and Discriminative Labeling for Multi-Label Active Learning Based on Maximum Correntropy Criterion , 2017, IEEE Transactions on Image Processing.

[19]  Lei Shi,et al.  Robust Multiple Kernel K-means Using L21-Norm , 2015, IJCAI.

[20]  Yung-Yu Chuang,et al.  Affinity aggregation for spectral clustering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Thomas Brox,et al.  Motion Segmentation & Multiple Object Tracking by Correlation Co-Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[23]  Jing-Hao Xue,et al.  On the orthogonal distance to class subspaces for high-dimensional data classification , 2017, Inf. Sci..

[24]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[25]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[26]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Mahardhika Pratama,et al.  Fuzzy clustering based nonlinear system identification and controller development of Pixhawk based quadcopter , 2017, 2017 Ninth International Conference on Advanced Computational Intelligence (ICACI).

[28]  Ahmet Bugra Koku,et al.  Unsupervised deep learning for subspace clustering , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[29]  Zenglin Xu,et al.  Unified Spectral Clustering with Optimal Graph , 2017, AAAI.

[30]  Zenglin Xu,et al.  An Extended Level Method for Efficient Multiple Kernel Learning , 2008, NIPS.

[31]  Jie Cao,et al.  GLEAM: a graph clustering framework based on potential game optimization for large-scale social networks , 2017, Knowledge and Information Systems.

[32]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[33]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[34]  Shuicheng Yan,et al.  Correntropy Induced L2 Graph for Robust Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[36]  Zhenwen Ren,et al.  Robust low-rank kernel multi-view subspace clustering based on the Schatten p-norm and correntropy , 2019, Inf. Sci..

[37]  Jesús Bobadilla,et al.  Recommender Systems Clustering Using Bayesian Non Negative Matrix Factorization , 2018, IEEE Access.

[38]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[39]  Shijian Lu,et al.  YoTube: Searching Action Proposal Via Recurrent and Static Regression Networks , 2017, IEEE Transactions on Image Processing.

[40]  Feiping Nie,et al.  A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering , 2015, IJCAI.

[41]  Jian Yang,et al.  Probabilistic Diffusion for Interactive Image Segmentation , 2019, IEEE Transactions on Image Processing.

[42]  Ran He,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Shifei Ding,et al.  A semi-supervised approximate spectral clustering algorithm based on HMRF model , 2018, Inf. Sci..

[44]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[45]  Wei-Yun Yau,et al.  Structured AutoEncoders for Subspace Clustering , 2018, IEEE Transactions on Image Processing.

[46]  Larry S. Davis,et al.  Learning Structured Low-Rank Representations for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Yung-Yu Chuang,et al.  Multiple Kernel Fuzzy Clustering , 2012, IEEE Transactions on Fuzzy Systems.