DealMVC: Dual Contrastive Calibration for Multi-view Clustering

Benefiting from the strong view-consistent information mining capacity, multi-view contrastive clustering has attracted plenty of attention in recent years. However, we observe the following drawback, which limits the clustering performance from further improvement. The existing multi-view models mainly focus on the consistency of the same samples in different views while ignoring the circumstance of similar but different samples in cross-view scenarios. To solve this problem, we propose a novel Dual contrastive calibration network for Multi-View Clustering (DealMVC). Specifically, we first design a fusion mechanism to obtain a global cross-view feature. Then, a global contrastive calibration loss is proposed by aligning the view feature similarity graph and the high-confidence pseudo-label graph. Moreover, to utilize the diversity of multi-view information, we propose a local contrastive calibration loss to constrain the consistency of pair-wise view features. The feature structure is regularized by reliable class information, thus guaranteeing similar samples have similar features in different views. During the training procedure, the interacted cross-view feature is jointly optimized at both local and global levels. In comparison with other state-of-the-art approaches, the comprehensive experimental results obtained from eight benchmark datasets provide substantial validation of the effectiveness and superiority of our algorithm. We release the code of DealMVC at https://github.com/xihongyang1999/DealMVC on GitHub.

[1]  Siwei Wang,et al.  Unpaired Multi-View Graph Clustering with Cross-View Structure Matching , 2023, IEEE transactions on neural networks and learning systems.

[2]  Stan Z. Li,et al.  OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning , 2023, ArXiv.

[3]  Siwei Wang,et al.  One-step Multi-view Clustering with Diverse Representation , 2023, ArXiv.

[4]  Xinwang Liu,et al.  Fast Continual Multi-View Clustering With Incomplete Views , 2023, IEEE Transactions on Image Processing.

[5]  Stan Z. Li,et al.  Dink-Net: Neural Clustering on Large Graphs , 2023, ICML.

[6]  Siwei Wang,et al.  Deep Incomplete Multi-View Clustering with Cross-View Partial Sample and Prototype Alignment , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Xiaobo Shen,et al.  Hierarchical Hashing Learning for Image Set Classification , 2023, IEEE Transactions on Image Processing.

[8]  Zhe Liu,et al.  Auto-weighted Multi-view Clustering for Large-scale Data , 2023, AAAI.

[9]  Yue Liu,et al.  Cluster-guided Contrastive Graph Clustering Network , 2023, AAAI.

[10]  Jinzhu Yang,et al.  Image Quality Assessment Guided Collaborative Learning of Image Enhancement and Classification for Diabetic Retinopathy Grading , 2022, IEEE Journal of Biomedical and Health Informatics.

[11]  Yue Liu,et al.  Hard Sample Aware Network for Contrastive Deep Graph Clustering , 2022, AAAI.

[12]  Xin Niu,et al.  Fast Incomplete Multi-View Clustering With View-Independent Anchors. , 2022, IEEE transactions on neural networks and learning systems.

[13]  Yue Liu,et al.  Contrastive Deep Graph Clustering with Learnable Augmentation , 2022, ArXiv.

[14]  Stan Z. Li,et al.  SimVP: Towards Simple yet Powerful Spatiotemporal Predictive Learning , 2022, ArXiv.

[15]  Dezhong Peng,et al.  Feature and Semantic Views Consensus Hashing for Image Set Classification , 2022, ACM Multimedia.

[16]  Xinwang Liu,et al.  Continual Multi-view Clustering , 2022, ACM Multimedia.

[17]  Xinwang Liu SimpleMKKM: Simple Multiple Kernel K-Means , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Yilong Yin,et al.  Not All Parameters Should Be Treated Equally: Deep Safe Semi-supervised Learning under Class Distribution Mismatch , 2022, AAAI.

[19]  Suyuan Liu,et al.  Efficient One-Pass Multi-View Subspace Clustering with Consensus Anchors , 2022, AAAI.

[20]  Yue Liu,et al.  Mixed Graph Contrastive Network for Semi-Supervised Node Classification , 2022, ACM Transactions on Knowledge Discovery from Data.

[21]  Huayi Tang,et al.  Deep Safe Multi-view Clustering: Reducing the Risk of Clustering Performance Degradation Caused by View Increase , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Stan Z. Li,et al.  Hyperspherical Consistency Regularization , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Yilong Yin,et al.  Safe-Student for Safe Deep Semi-Supervised Learning with Unseen-Class Unlabeled Data , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Xinzhong Zhu,et al.  Align then Fusion: Generalized Large-scale Multi-view Clustering with Anchor Matching Correspondences , 2022, NeurIPS.

[25]  Siwei Wang,et al.  Simple Contrastive Graph Clustering , 2022, IEEE transactions on neural networks and learning systems.

[26]  Hong Wang,et al.  Collaborative learning of weakly-supervised domain adaptation for diabetic retinopathy grading on retinal images , 2022, Comput. Biol. Medicine.

[27]  Yue Liu,et al.  Improved Dual Correlation Reduction Network , 2022, ArXiv.

[28]  En Zhu,et al.  Interpolation-based Contrastive Learning for Few-Label Semi-Supervised Learning , 2022, IEEE transactions on neural networks and learning systems.

[29]  En Zhu,et al.  Deep Graph Clustering via Dual Correlation Reduction , 2021, AAAI.

[30]  Xinzhong Zhu,et al.  Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance , 2021, IEEE Transactions on Image Processing.

[31]  Kun Zhan,et al.  Stationary Diffusion State Neural Estimation for Multiview Clustering , 2021, AAAI.

[32]  En Zhu,et al.  Late Fusion Multiple Kernel Clustering With Proxy Graph Refinement , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Xinwang Liu Incomplete Multiple Kernel Alignment Maximization for Clustering. , 2021, IEEE transactions on pattern analysis and machine intelligence.

[34]  Lirong Wu,et al.  Co-learning: Learning from Noisy Labels with Self-supervision , 2021, ACM Multimedia.

[35]  Yazhou Ren,et al.  Multi-level Feature Learning for Contrastive Multi-view Clustering , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yazhou Ren,et al.  Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Xi Peng,et al.  COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  En Zhu,et al.  Consensus Graph Learning for Multi-View Clustering , 2021, IEEE Transactions on Multimedia.

[39]  Yazhou Ren,et al.  Self-Supervised Discriminative Feature Learning for Deep Multi-View Clustering , 2021, IEEE Transactions on Knowledge and Data Engineering.

[40]  Michael Kampffmeyer,et al.  Reconsidering Representation Alignment for Multi-view Clustering , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Pei Zhang,et al.  Adaptive Weighted Graph Fusion Incomplete Multi-View Subspace Clustering , 2020, Sensors.

[42]  Zenglin Xu,et al.  Deep Embedded Multi-view Clustering with Collaborative Training , 2020, Inf. Sci..

[43]  Qianqian Wang,et al.  Multi-View Attribute Graph Convolution Networks for Clustering , 2020, IJCAI.

[44]  Yi-Dong Shen,et al.  End-to-End Adversarial-Attention Network for Multi-Modal Clustering , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[46]  Yang Wang,et al.  SimpleMKKM: Simple Multiple Kernel K-means , 2020, ArXiv.

[47]  Chang Tang,et al.  CGD: Multi-View Clustering via Cross-View Graph Diffusion , 2020, AAAI.

[48]  Xinwang Liu,et al.  Multiple Kernel Clustering With Neighbor-Kernel Subspace Segmentation , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Chang Tang,et al.  Efficient and Effective Regularized Incomplete Multi-View Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[51]  En Zhu,et al.  Subspace segmentation-based robust multiple kernel clustering , 2020, Inf. Fusion.

[52]  Zenglin Xu,et al.  Auto-weighted multi-view clustering via deep matrix decomposition , 2020, Pattern Recognit..

[53]  Ross B. Girshick,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Changqing Zhang,et al.  Multi-view Deep Subspace Clustering Networks , 2019, ArXiv.

[55]  Zhaoyang Li,et al.  Deep Adversarial Multi-view Clustering Network , 2019, IJCAI.

[56]  Phillip Isola,et al.  Contrastive Multiview Coding , 2019, ECCV.

[57]  Huazhu Fu,et al.  AE2-Nets: Autoencoder in Autoencoder Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Jiancheng Lv,et al.  COMIC: Multi-view Clustering Without Parameter Selection , 2019, ICML.

[59]  Jian Li,et al.  Multi-Head Attention with Disagreement Regularization , 2018, EMNLP.

[60]  Yong Xu,et al.  Incomplete Multi-view Clustering via Graph Regularized Matrix Factorization , 2018, ECCV Workshops.

[61]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[62]  Meng Wang,et al.  Self-Supervised Video Hashing With Hierarchical Binary Auto-Encoder , 2018, IEEE Transactions on Image Processing.

[63]  Yun Fu,et al.  Multi-View Clustering via Deep Matrix Factorization , 2017, AAAI.

[64]  Xiaochun Cao,et al.  Diversity-induced Multi-view Subspace Clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[66]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[67]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[68]  Aapo Hyvärinen,et al.  Estimation of Non-Normalized Statistical Models by Score Matching , 2005, J. Mach. Learn. Res..

[69]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[70]  Xin Xu,et al.  Patch-Mixing Contrastive Regularization for Few-Label Semi-Supervised Learning , 2023, IEEE Transactions on Artificial Intelligence.

[71]  Xu Wang,et al.  Hierarchical Consensus Hashing for Cross-Modal Retrieval , 2023, IEEE transactions on multimedia.

[72]  En Zhu,et al.  One Pass Late Fusion Multi-view Clustering , 2021, ICML.

[73]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[74]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .