Self-weighted multi-view clustering with soft capped norm

Abstract Real-world data sets are often comprised of multiple representations or modalities which provide different and complementary aspects of information. Multi-view clustering plays an indispensable role in analyzing multi-view data. In multi-view learning, one key step is assigning a reasonable weight to each view according to the view importance. Most existing work learn the weights by introducing a hyperparameter, which is undesired in practice. In this paper, our proposed model learns an optimal weight for each view automatically without introducing an additive parameter as previous methods do. Furthermore, to deal with different level noises and outliers, we propose to use ‘soft’ capped norm, which caps the residual of outliers as a constant value and provides a probability for certain data point being an outlier. An efficient updating algorithm is designed to solve our model and its convergence is also guaranteed theoretically. Extensive experimental results on several real-world data sets show that our proposed model outperforms state-of-the-art multi-view clustering algorithms.

[1]  Zenglin Xu,et al.  Bayesian Nonparametric Models for Multiway Data Analysis , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[3]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[4]  Ying Cui,et al.  Non-redundant Multi-view Clustering via Orthogonalization , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[5]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[6]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[7]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Dingcheng Li,et al.  Spectral co-clustering ensemble , 2015, Knowl. Based Syst..

[9]  Zenglin Xu,et al.  Association Discovery and Diagnosis of Alzheimer's Disease with Bayesian Multiview Learning , 2016, J. Artif. Intell. Res..

[10]  Xingpeng Jiang,et al.  Multi-View Clustering of Microbiome Samples by Robust Similarity Network Fusion and Spectral Clustering , 2017, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[11]  许超 Multi-View Self-Paced Learning for Clustering , 2015 .

[12]  José Carlos Príncipe,et al.  Information Theoretic Clustering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Mingjing Li,et al.  Color texture moments for content-based image retrieval , 2002, Proceedings. International Conference on Image Processing.

[14]  Robert Jenssen,et al.  Information theoretic clustering , 2010, Scholarpedia.

[15]  Yong Dou,et al.  Multi-view clustering with extreme learning machine , 2016, Neurocomputing.

[16]  Aristidis Likas,et al.  Kernel-Based Weighted Multi-view Clustering , 2012, 2012 IEEE 12th International Conference on Data Mining.

[17]  Yuhong Guo,et al.  Convex Subspace Representation Learning from Multi-View Data , 2013, AAAI.

[18]  Qingyao Wu,et al.  NMFE-SSCC: Non-negative matrix factorization ensemble for semi-supervised collective classification , 2015, Knowl. Based Syst..

[19]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[20]  Derek Greene,et al.  A Matrix Factorization Approach for Integrating Multiple Data Views , 2009, ECML/PKDD.

[21]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[22]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[23]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Weihua Ou,et al.  Multi-view non-negative matrix factorization by patch alignment framework with view consistency , 2016, Neurocomputing.

[25]  Xuelong Li,et al.  Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification , 2016, IJCAI.

[26]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[27]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[28]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  Zenglin Xu,et al.  Nonnegative matrix factorization with adaptive neighbors , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[30]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[31]  Zenglin Xu,et al.  Adaptive local structure learning for document co-clustering , 2018, Knowl. Based Syst..

[32]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[33]  Feiping Nie,et al.  Heterogeneous image feature integration via multi-modal spectral clustering , 2011, CVPR 2011.

[34]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[35]  Qi Xie,et al.  Self-Paced Learning for Matrix Factorization , 2015, AAAI.

[36]  Christoph H. Lampert,et al.  Correlational spectral clustering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Feiping Nie,et al.  Robust Capped Norm Nonnegative Matrix Factorization: Capped Norm NMF , 2015, CIKM.

[38]  Yun Fu,et al.  Multi-View Clustering via Deep Matrix Factorization , 2017, AAAI.

[39]  Zhao Kang,et al.  Kernel-driven similarity learning , 2017, Neurocomputing.

[40]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[41]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[42]  Gilles Bisson,et al.  Co-clustering of Multi-view Datasets: A Parallelizable Approach , 2012, 2012 IEEE 12th International Conference on Data Mining.

[43]  Zhao Kang,et al.  Robust Graph Regularized Nonnegative Matrix Factorization for Clustering , 2017, ACM Trans. Knowl. Discov. Data.

[44]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[45]  Zenglin Xu,et al.  Joint Association Discovery and Diagnosis of Alzheimer's Disease by Supervised Heterogeneous Multiview Learning , 2013, Pacific Symposium on Biocomputing.

[46]  Zenglin Xu,et al.  Infinite Tucker Decomposition: Nonparametric Bayesian Models for Multiway Data Analysis , 2011, ICML.

[47]  Zenglin Xu,et al.  Sparse Bayesian Multiview Learning for Simultaneous Association Discovery and Diagnosis of Alzheimer's Disease , 2015, AAAI.

[48]  Xianchao Zhang,et al.  Multi-Task Multi-View Clustering for Non-Negative Data , 2015, IJCAI.

[49]  Zi Huang,et al.  Discrete Nonnegative Spectral Clustering , 2017, IEEE Transactions on Knowledge and Data Engineering.

[50]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[51]  Feiping Nie,et al.  Discriminatively Embedded K-Means for Multi-view Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Zenglin Xu,et al.  Robust multi-view data clustering with multi-view capped-norm K-means , 2018, Neurocomputing.

[53]  Yang Yang,et al.  Multitask Spectral Clustering by Exploring Intertask Correlation , 2015, IEEE Transactions on Cybernetics.

[54]  Hong Yu,et al.  Constrained NMF-Based Multi-View Clustering on Unmapped Data , 2015, AAAI.

[55]  V. D. Sa Spectral Clustering with Two Views , 2007 .

[56]  Liang Wang,et al.  Multi-view clustering via pairwise sparse subspace representation , 2015, Neurocomputing.

[57]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[58]  Hong Yu,et al.  Local linear neighbor reconstruction for multi-view data , 2016, Pattern Recognit. Lett..

[59]  Hong Yu,et al.  Multi-view clustering via multi-manifold regularized non-negative matrix factorization , 2017, Neural Networks.