Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering

Ensemble clustering (EC), utilizing multiple basic partitions (BPs) to yield a robust consensus clustering, has shown promising clustering performance. Nevertheless, most current algorithms suffer from two challenging hurdles: (1) a surge of EC-based methods only focus on pair-wise sample correlation while fully ignoring the high-order correlations of diverse views. (2) they deal directly with the co-association (CA) matrices generated from BPs, which are inevitably corrupted by noise and thus degrade the clustering performance. To address these issues, we propose a novel Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering (DC-RMEC) method, which preserves the high-order inter-view correlation and the high-order correlation of original data simultaneously. Specifically, DC-RMEC constructs a hypergraph from BPs to fuse high-level complementary information from different algorithms and incorporates multiple CA-based representations into a low-rank tensor to discover the high-order relevance underlying CA matrices, such that double high-order correlation of multi-view features could be dexterously uncovered. Moreover, a marginalized denoiser is invoked to gain robust view-specific CA matrices. Furthermore, we develop a unified framework to jointly optimize the representation tensor and the result matrix. An effective iterative optimization algorithm is designed to optimize our DC-RMEC model by resorting to the alternating direction method of multipliers. Extensive experiments on seven real-world multi-view datasets have demonstrated the superiority of DC-RMEC compared with several state-of-the-art multi-view ensemble clustering methods.

[1]  Changdong Wang,et al.  Fast Multi-View Clustering Via Ensembles: Towards Scalability, Superiority, and Simplicity , 2022, IEEE Transactions on Knowledge and Data Engineering.

[2]  Xinzhong Zhu,et al.  Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance , 2021, IEEE Transactions on Image Processing.

[3]  Hong Jia,et al.  Spectral Ensemble Clustering with LDA-based Co-training for Multi-view Data Analysis , 2021, 2021 17th International Conference on Computational Intelligence and Security (CIS).

[4]  En Zhu,et al.  Scalable Multi-view Subspace Clustering with Unified Anchors , 2021, ACM Multimedia.

[5]  Y. Fu,et al.  From Ensemble Clustering to Subspace Clustering: Cluster Structure Encoding , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Chong Peng,et al.  Low-Rank Tensor Graph Learning for Multi-View Subspace Clustering , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  T. Sakurai,et al.  Ensemble Learning for Spectral Clustering , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[8]  Sudhish N. George,et al.  A Unified Tensor Framework for Clustering and Simultaneous Reconstruction of Incomplete Imaging Data , 2020, ACM Trans. Multim. Comput. Commun. Appl..

[9]  Hao Wang,et al.  GMC: Graph-Based Multi-View Clustering , 2020, IEEE Transactions on Knowledge and Data Engineering.

[10]  Yun Fu,et al.  Marginalized Multiview Ensemble Clustering , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Richang Hong,et al.  Robust Subspace Discovery by Block-diagonal Adaptive Locality-constrained Representation , 2019, ACM Multimedia.

[12]  Yun Fu,et al.  Adversarial Graph Embedding for Ensemble Clustering , 2019, IJCAI.

[13]  Xuelong Li,et al.  Flexible Affinity Matrix Learning for Unsupervised and Semisupervised Classification , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Qi Tian,et al.  Discovering Latent Topics by Gaussian Latent Dirichlet Allocation and Spectral Clustering , 2019, ACM Trans. Multim. Comput. Commun. Appl..

[15]  Yun Fu,et al.  Robust Spectral Ensemble Clustering via Rank Minimization , 2019, ACM Trans. Knowl. Discov. Data.

[16]  Chang-Dong Wang,et al.  Enhanced Ensemble Clustering via Fast Propagation of Cluster-Wise Similarities , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Yun Fu,et al.  Consensus Guided Multi-View Clustering , 2018, ACM Trans. Knowl. Discov. Data.

[18]  Yang Wang,et al.  Multiview Spectral Clustering via Structured Low-Rank Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Ivica Kopriva,et al.  Multi-view low-rank sparse subspace clustering , 2017, Pattern Recognit..

[20]  Ming Shao,et al.  Infinite ensemble clustering , 2017, Data Mining and Knowledge Discovery.

[21]  Lin Wu,et al.  Beyond Low-Rank Representations: Orthogonal Clustering Basis Reconstruction with Optimized Graph Structure for Multi-view Spectral Clustering , 2017, Neural Networks.

[22]  Yun Fu,et al.  From Ensemble Clustering to Multi-View Clustering , 2017, IJCAI.

[23]  Junjie Wu,et al.  Spectral Ensemble Clustering via Weighted K-Means: Theoretical and Practical Evidence , 2017, IEEE Transactions on Knowledge and Data Engineering.

[24]  Yun Fu,et al.  Robust Spectral Ensemble Clustering , 2016, CIKM.

[25]  Yuan Xie,et al.  On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization , 2016, International Journal of Computer Vision.

[26]  Panagiotis Symeonidis,et al.  ClustHOSVD: Item Recommendation by Combining Semantically Enhanced Tag Clustering With Tensor HOSVD , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Ming Shao,et al.  Infinite Ensemble for Image Clustering , 2016, KDD.

[28]  Lin Wu,et al.  Iterative Views Agreement: An Iterative Low-Rank Based Structured Optimization Method to Multi-View Spectral Clustering , 2016, IJCAI.

[29]  Xuelong Li,et al.  Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification , 2016, IJCAI.

[30]  Chang-Dong Wang,et al.  Locally Weighted Ensemble Clustering , 2016, IEEE Transactions on Cybernetics.

[31]  Chang-Dong Wang,et al.  Robust Ensemble Clustering Using Probability Trajectories , 2016, IEEE Transactions on Knowledge and Data Engineering.

[32]  Yang Wang,et al.  Shifting multi-hypergraphs via collaborative probabilistic voting , 2016, Knowledge and Information Systems.

[33]  Chang-Dong Wang,et al.  Ensemble clustering using factor graph , 2016, Pattern Recognit..

[34]  Yun Fu,et al.  Clustering with Partition Level Side Information , 2015, 2015 IEEE International Conference on Data Mining.

[35]  Junjie Wu,et al.  Spectral Ensemble Clustering , 2015, KDD.

[36]  Lei Shi,et al.  Learning a Robust Consensus Matrix for Clustering Ensemble via Kullback-Leibler Divergence Minimization , 2015, IJCAI.

[37]  Feiping Nie,et al.  Large-Scale Multi-View Spectral Clustering via Bipartite Graph , 2015, AAAI.

[38]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.

[39]  Xiaoyi Jiang,et al.  Ensemble clustering by means of clustering embedding in vector spaces , 2014, Pattern Recognit..

[40]  Petros Daras,et al.  The TFC Model: Tensor Factorization and Tag Clustering for Item Recommendation in Social Tagging Systems , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[41]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[42]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[43]  Yu-Chiang Frank Wang,et al.  Low-rank matrix recovery with structural incoherence for robust face recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[45]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[46]  Tossapon Boongoen,et al.  A Link-Based Approach to the Cluster Ensemble Problem , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  René Vidal,et al.  Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[50]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[51]  Xiaoyu Wang,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[52]  C. Schnörr,et al.  Spectral clustering of linear subspaces for motion segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[53]  Hui Xiong,et al.  Adapting the right measures for K-means clustering , 2009, KDD.

[54]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[55]  Ana L. N. Fred,et al.  Combining multiple clusterings using evidence accumulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Carla E. Brodley,et al.  Solving cluster ensemble problems by bipartite graph partitioning , 2004, ICML.

[57]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[58]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[59]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[60]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[61]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[62]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[63]  Hui Xiong,et al.  K-Means-Based Consensus Clustering: A Unified View , 2015, IEEE Transactions on Knowledge and Data Engineering.

[64]  Dan A. Simovici,et al.  Finding Median Partitions Using Information-Theoretical-Based Genetic Algorithms , 2002, J. Univers. Comput. Sci..