Consensus Guided Multi-View Clustering

In recent decades, tremendous emerging techniques thrive the artificial intelligence field due to the increasing collected data captured from multiple sensors. These multi-view data provide more rich information than traditional single-view data. Fusing heterogeneous information for certain tasks is a core part of multi-view learning, especially for multi-view clustering. Although numerous multi-view clustering algorithms have been proposed, most scholars focus on finding the common space of different views, but unfortunately ignore the benefits from partition level by ensemble clustering. For ensemble clustering, however, there is no interaction between individual partitions from each view and the final consensus one. To fill the gap, we propose a Consensus Guided Multi-View Clustering (CMVC) framework, which incorporates the generation of basic partitions from each view and fusion of consensus clustering in an interactive way, i.e., the consensus clustering guides the generation of basic partitions, and high quality basic partitions positively contribute to the consensus clustering as well. We design a non-trivial optimization solution to formulate CMVC into two iterative k-means clusterings by an approximate calculation. In addition, the generalization of CMVC provides a rich feasibility for different scenarios, and the extension of CMVC with incomplete multi-view clustering further validates the effectiveness for real-world applications. Extensive experiments demonstrate the advantages of CMVC over other widely used multi-view clustering methods in terms of cluster validity, and the robustness of CMVC to some important parameters and incomplete multi-view data.

[1]  Ana L. N. Fred,et al.  Combining multiple clusterings using evidence accumulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Shao-Yuan Li,et al.  Partial Multi-View Clustering , 2014, AAAI.

[3]  Yun Fu,et al.  Simultaneous Clustering and Ensemble , 2017, AAAI.

[4]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[5]  Hui Xiong,et al.  K-Means-Based Consensus Clustering: A Unified View , 2015, IEEE Transactions on Knowledge and Data Engineering.

[6]  Ming Shao,et al.  Infinite ensemble clustering , 2017, Data Mining and Knowledge Discovery.

[7]  Hui Xiong,et al.  A Theoretic Framework of K-Means-Based Consensus Clustering , 2013, IJCAI.

[8]  Hui Xiong,et al.  A Generalization of Distance Functions for Fuzzy $c$ -Means Clustering With Centroids of Arithmetic Means , 2012, IEEE Transactions on Fuzzy Systems.

[9]  Carla E. Brodley,et al.  Solving cluster ensemble problems by bipartite graph partitioning , 2004, ICML.

[10]  L. Carin,et al.  Analytical Kernel Matrix Completion with Incomplete Multi-View Data , 2005 .

[11]  Philip S. Yu,et al.  Multiple Incomplete Views Clustering via Weighted Nonnegative Matrix Factorization with L2, 1 Regularization , 2015, ECML/PKDD.

[12]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[13]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[14]  Junjie Wu,et al.  Spectral Ensemble Clustering via Weighted K-Means: Theoretical and Practical Evidence , 2017, IEEE Transactions on Knowledge and Data Engineering.

[15]  Fei Wang,et al.  Multi-View Local Learning , 2008, AAAI.

[16]  Ying Cui,et al.  Non-redundant Multi-view Clustering via Orthogonalization , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[17]  Yun Fu,et al.  Partition Level Constrained Clustering , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Hong Zhou,et al.  Accurate integration of multi-view range images using k-means clustering , 2008, Pattern Recognit..

[19]  Ming Shao,et al.  Infinite Ensemble for Image Clustering , 2016, KDD.

[20]  Jieping Ye,et al.  Multi-objective Multi-view Spectral Clustering via Pareto Optimization , 2013, SDM.

[21]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[22]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[23]  Yun Fu,et al.  From Ensemble Clustering to Multi-View Clustering , 2017, IJCAI.

[24]  Junjie Wu,et al.  Spectral Ensemble Clustering , 2015, KDD.

[25]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.

[26]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[27]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[28]  Stéphane Marchand-Maillet,et al.  Multiview clustering: a late fusion approach using latent models , 2009, SIGIR.

[29]  Geoffrey J. Gordon,et al.  Relational learning via collective matrix factorization , 2008, KDD.

[30]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[31]  Junjie Wu,et al.  DIAS: A Disassemble-Assemble Framework for Highly Sparse Text Clustering , 2015, SDM.

[32]  Yun Fu,et al.  Incomplete Multi-Modal Visual Data Grouping , 2016, IJCAI.

[33]  Yun Fu,et al.  Clustering with Partition Level Side Information , 2015, 2015 IEEE International Conference on Data Mining.

[34]  Hui Xiong,et al.  Adapting the right measures for K-means clustering , 2009, KDD.

[35]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[36]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[37]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[38]  Yun Fu,et al.  Robust Spectral Ensemble Clustering , 2016, CIKM.

[39]  Christoph H. Lampert,et al.  Correlational spectral clustering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Massih-Reza Amini,et al.  Multi-view clustering of multilingual documents , 2010, SIGIR.

[41]  Yongdong Zhang,et al.  Multiview Spectral Embedding , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Boris G. Mirkin,et al.  Reinterpreting the Category Utility Function , 2001, Machine Learning.

[43]  Eric Eaton,et al.  Multi-view clustering with constraint propagation for learning with an incomplete mapping between views , 2010, CIKM.

[44]  Juho Rousu,et al.  Multi-view kernel completion , 2016, Machine Learning.

[45]  Ning Chen,et al.  Predictive Subspace Learning for Multi-view Data: a Large Margin Approach , 2010, NIPS.

[46]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[47]  Yuhong Guo,et al.  Convex Subspace Representation Learning from Multi-View Data , 2013, AAAI.