Efficient Orthogonal Multi-view Subspace Clustering

Multi-view subspace clustering targets at clustering data lying in a union of low-dimensional subspaces. Generally, an n X n affinity graph is constructed, on which spectral clustering is then performed to achieve the final clustering. Both graph construction and graph partitioning of spectral clustering suffer from quadratic or even cubic time and space complexity, leading to difficulty in clustering large-scale datasets. Some efforts have recently been made to capture data distribution in multiple views by selecting key anchor bases beforehand with k-means or uniform sampling strategy. Nevertheless, few of them pay attention to the algebraic property of the anchors. How to learn a set of high-quality orthogonal bases in a unified framework, while maintaining its scalability for very large datasets, remains a big challenge. In view of this, we propose an Efficient Orthogonal Multi-view Subspace Clustering (OMSC) model with almost linear complexity. Specifically, the anchor learning, graph construction and partition are jointly modeled in a unified framework. With the mutual enhancement of each other, a more discriminative and flexible anchor representation and cluster indicator can be jointly obtained. An alternate minimizing strategy is developed to deal with the optimization problem, which is proved to have linear time complexity w.r.t. the sample number. Extensive experiments have been conducted to confirm the superiority of the proposed OMSC method. The source codes and data are available at https://github.com/ManshengChen/Code-for-OMSC-master.

[1]  Changdong Wang,et al.  Fast Multi-View Clustering Via Ensembles: Towards Scalability, Superiority, and Simplicity , 2022, IEEE Transactions on Knowledge and Data Engineering.

[2]  Xinzhong Zhu,et al.  Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance , 2021, IEEE Transactions on Image Processing.

[3]  En Zhu,et al.  Scalable Multi-view Subspace Clustering with Unified Anchors , 2021, ACM Multimedia.

[4]  Hong Yu,et al.  A Novel Multi-View Clustering Method for Unknown Mapping Relationships Between Cross-View Samples , 2021, KDD.

[5]  Lihua Zhou,et al.  Deep Multiple Auto-Encoder-Based Multi-view Clustering , 2021, Data Science and Engineering.

[6]  Chang-Dong Wang,et al.  Relaxed multi-view clustering in latent embedding space , 2021, Inf. Fusion.

[7]  Feiping Nie,et al.  Fast Multi-View Clustering via Nonnegative and Orthogonal Factorization , 2020, IEEE Transactions on Image Processing.

[8]  Philip S. Yu,et al.  Smoothness Regularized Multiview Subspace Clustering With Kernel Learning , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Philip S. Yu,et al.  MVStream: Multiview Data Stream Clustering , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Hao Wang,et al.  GMC: Graph-Based Multi-View Clustering , 2020, IEEE Transactions on Knowledge and Data Engineering.

[11]  Chang-Dong Wang,et al.  Multi-View Clustering in Latent Embedding Space , 2020, AAAI.

[12]  Feiping Nie,et al.  Auto-weighted multi-view co-clustering via fast matrix factorization , 2020, Pattern Recognit..

[13]  Qinghua Hu,et al.  Generalized Latent Multi-View Subspace Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Zenglin Xu,et al.  Large-scale Multi-view Subspace Clustering in Linear Time , 2019, AAAI.

[15]  Zenglin Xu,et al.  Partition level multiview subspace clustering , 2019, Neural Networks.

[16]  Dinggang Shen,et al.  Late Fusion Incomplete Multi-View Clustering , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Zhihui Li,et al.  Spectral Clustering of Customer Transaction Data With a Two-Level Subspace Weighting Method , 2019, IEEE Transactions on Cybernetics.

[18]  En Zhu,et al.  Multi-view Clustering via Late Fusion Alignment Maximization , 2019, IJCAI.

[19]  Xinwang Liu,et al.  Learning a Joint Affinity Graph for Multiview Subspace Clustering , 2019, IEEE Transactions on Multimedia.

[20]  Stan Z. Li,et al.  Multi-view subspace clustering with intactness-aware similarity , 2019, Pattern Recognit..

[21]  Shiliang Sun,et al.  Multiview Machine Learning , 2019, Springer Singapore.

[22]  Zenglin Xu,et al.  Low-rank kernel learning for graph-based clustering , 2019, Knowl. Based Syst..

[23]  Hongchuan Yu,et al.  Diverse Non-Negative Matrix Factorization for Multiview Data Representation , 2018, IEEE Transactions on Cybernetics.

[24]  Wei Zhang,et al.  Consistent and Specific Multi-View Subspace Clustering , 2018, AAAI.

[25]  Xuelong Li,et al.  Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification , 2018, IEEE Transactions on Image Processing.

[26]  Chang-Dong Wang,et al.  Multi-view Intact Space Clustering , 2017, 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR).

[27]  Feiping Nie,et al.  Multi-View Unsupervised Feature Selection with Adaptive Similarity and View Weight , 2017, IEEE Transactions on Knowledge and Data Engineering.

[28]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[29]  Stan Z. Li,et al.  Exclusivity-Consistency Regularized Multi-view Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Qinghua Hu,et al.  Latent Multi-view Subspace Clustering , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Xuelong Li,et al.  Bilateral k-Means Algorithm for Fast Co-Clustering , 2017, AAAI.

[32]  Yuan Xie,et al.  On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization , 2016, International Journal of Computer Vision.

[33]  Xuelong Li,et al.  Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification , 2016, IJCAI.

[34]  Philip S. Yu,et al.  Multi-View Clustering Based on Belief Propagation , 2016, IEEE Transactions on Knowledge and Data Engineering.

[35]  Xiaochun Cao,et al.  Low-Rank Tensor Constrained Multiview Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Xuelong Li,et al.  Multi-view Subspace Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Xiaochun Cao,et al.  Constrained Multi-View Video Face Clustering , 2015, IEEE Transactions on Image Processing.

[38]  Xiaochun Cao,et al.  Diversity-induced Multi-view Subspace Clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[40]  Jianjiang Feng,et al.  Smooth Representation Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[42]  Qiang Ji,et al.  Constrained Clustering and Its Application to Face Clustering in Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[44]  Fillia Makedon,et al.  Fast Nonnegative Matrix Tri-Factorization for Large-Scale Data Co-Clustering , 2011, IJCAI.

[45]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[47]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[48]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.