Robust multi-view clustering via inter-and-intra-view low rank fusion

Abstract Multi-view Clustering has become a vital task with the rapidly growing amount of data in multiple representations. Under the context of unsupervised learning, the existing clustering methods show limited capability in leveraging the discriminative information from multiple views. View-independent noises and distribution differences among views are two main causes that may degrade the clustering performances. By introducing the inter-and-intra-view low rank decomposition, we propose Multi-view Clustering via Intra and Inter-view Low Rank Fusion (MCIIF). The proposed method facilitates clustering with the discriminative intra-view low rank structures that complement the shared inter-view transition. By recovering complementary low rank transitions from the differences between the shared inter-view transition and individual-view transitions, we pursue a more accurate transition probability matrix by combining the complementary intra-view transitions with the shared inter-view transition in an additive manner. To solve the corresponding optimization problem, we propose a procedure based on the Alternating Direction Method of Multipliers (ADMM) scheme with convergence guarantees. Experimental results on various real-world datasets verify the effectiveness of the proposed method over the state-of-art multi-view clustering methods.

[1]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[2]  Yehuda Koren,et al.  Lessons from the Netflix prize challenge , 2007, SKDD.

[3]  Alexander J. Smola,et al.  Explaining Reviews and Ratings with PACO: Poisson Additive Co-Clustering , 2015, WWW.

[4]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[5]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[6]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[7]  Roger Levy,et al.  On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Zibin Zheng,et al.  Auto-weighted multi-view constrained spectral clustering , 2019, Neurocomputing.

[9]  Bo Du,et al.  Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding , 2015, Pattern Recognit..

[10]  Tomislav Šmuc,et al.  The landscape of microbial phenotypic traits and associated genes , 2016, Nucleic acids research.

[11]  Shuicheng Yan,et al.  Convex Sparse Spectral Clustering: Single-View to Multi-View , 2015, IEEE Transactions on Image Processing.

[12]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[13]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[14]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[15]  Xiaochun Cao,et al.  Low-Rank Tensor Constrained Multiview Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[17]  Wei Zhang,et al.  Consistent and Specific Multi-View Subspace Clustering , 2018, AAAI.

[18]  Lise Getoor,et al.  Link-Based Classification , 2003, Encyclopedia of Machine Learning and Data Mining.

[19]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[20]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[21]  Stan Z. Li,et al.  Exclusivity-Consistency Regularized Multi-view Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Meng Wang,et al.  Multimodal Deep Autoencoder for Human Pose Recovery , 2015, IEEE Transactions on Image Processing.

[23]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[24]  Dong Yue,et al.  Multi-view low-rank dictionary learning for image classification , 2016, Pattern Recognit..

[25]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[26]  Heinz H. Bauschke,et al.  A Weak-to-Strong Convergence Principle for Fejé-Monotone Methods in Hilbert Spaces , 2001, Math. Oper. Res..

[27]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[28]  Emmanuel J. Candès,et al.  Robust Subspace Clustering , 2013, ArXiv.

[29]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[30]  Ivica Kopriva,et al.  Multi-view low-rank sparse subspace clustering , 2017, Pattern Recognit..

[31]  James Bennett,et al.  The Netflix Prize , 2007 .

[32]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[33]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[34]  Yong Luo,et al.  Low-Rank Multi-View Learning in Matrix Completion for Multi-Label Image Classification , 2015, AAAI.

[35]  Qian Tao,et al.  An intelligent clustering algorithm for high-dimensional multiview data in big data applications , 2020, Neurocomputing.

[36]  Xin Zuo,et al.  Auto-encoder Based Co-training Multi-view Representation Learning , 2019, PAKDD.

[37]  Min Xie,et al.  CCCF: Improving Collaborative Filtering via Scalable User-Item Co-Clustering , 2016, WSDM.

[38]  Kun Zhan,et al.  Graph Learning for Multiview Clustering , 2018, IEEE Transactions on Cybernetics.

[39]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[40]  Roger Levy,et al.  A new approach to cross-modal multimedia retrieval , 2010, ACM Multimedia.

[41]  Fan Ye,et al.  Incremental multi-view spectral clustering , 2019, Knowl. Based Syst..

[42]  Xuelong Li,et al.  Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification , 2016, IJCAI.

[43]  Xin Wang,et al.  Robust Auto-Weighted Multi-View Clustering , 2018, IJCAI.

[44]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[45]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[46]  Hongbin Zha,et al.  Essential Tensor Learning for Multi-View Spectral Clustering , 2018, IEEE Transactions on Image Processing.

[47]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[49]  Bernhard Schölkopf,et al.  Learning from labeled and unlabeled data on a directed graph , 2005, ICML.

[50]  Lin Wu,et al.  Multiview Spectral Clustering via Structured Low-Rank Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[52]  Alexander J. Smola,et al.  ACCAMS: Additive Co-Clustering to Approximate Matrices Succinctly , 2014, WWW.

[53]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[54]  Huafeng Liu,et al.  Additive Co-Clustering with Social Influence for Recommendation , 2017, RecSys.

[55]  Feiping Nie,et al.  Heterogeneous image feature integration via multi-modal spectral clustering , 2011, CVPR 2011.

[56]  Yun Fu,et al.  Low-Rank Common Subspace for Multi-view Learning , 2014, 2014 IEEE International Conference on Data Mining.

[57]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.