CONVERT: Contrastive Graph Clustering with Reliable Augmentation

Contrastive graph node clustering via learnable data augmentation is a hot research spot in the field of unsupervised graph learning. The existing methods learn the sampling distribution of a pre-defined augmentation to generate data-driven augmentations automatically. Although promising clustering performance has been achieved, we observe that these strategies still rely on pre-defined augmentations, the semantics of the augmented graph can easily drift. The reliability of the augmented view semantics for contrastive learning can not be guaranteed, thus limiting the model performance. To address these problems, we propose a novel CONtrastiVe Graph ClustEring network with Reliable AugmenTation (COVERT). Specifically, in our method, the data augmentations are processed by the proposed reversible perturb-recover network. It distills reliable semantic information by recovering the perturbed latent embeddings. Moreover, to further guarantee the reliability of semantics, a novel semantic loss is presented to constrain the network via quantifying the perturbation and recovery. Lastly, a label-matching mechanism is designed to guide the model by clustering information through aligning the semantic labels and the selected high-confidence clustering pseudo labels. Extensive experimental results on seven datasets demonstrate the effectiveness of the proposed method. We release the code and appendix of CONVERT at https://github.com/xihongyang1999/CONVERT on GitHub.

[1]  Shifei Ding,et al.  Graph clustering network with structure embedding enhanced , 2023, Pattern Recognition.

[2]  Zaixin Zhang,et al.  A Systematic Survey in Geometric Deep Learning for Structure-based Drug Design , 2023, ArXiv.

[3]  Stan Z. Li,et al.  Dink-Net: Neural Clustering on Large Graphs , 2023, ICML.

[4]  Yue Liu,et al.  Message Intercommunication for Inductive Relation Reasoning , 2023, ArXiv.

[5]  Zaixin Zhang,et al.  Learning Subpocket Prototypes for Generalizable Structure-based Drug Design , 2023, ICML.

[6]  Chang-Yu Hsieh,et al.  An equivariant generative framework for molecular graph-structure Co-design , 2023, bioRxiv.

[7]  Xiaolei Huang,et al.  ABSLearn: a GNN-based framework for aliasing and buffer-size information retrieval , 2023, Pattern Analysis and Applications.

[8]  Xihong Yang,et al.  Self-Supervised Temporal Graph learning with Temporal and Structural Intensity Alignment , 2023, ArXiv.

[9]  Shirui Pan,et al.  Neighbor Contrastive Learning on Learnable Graph Augmentation , 2023, AAAI Conference on Artificial Intelligence.

[10]  Yue Liu,et al.  Cluster-guided Contrastive Graph Clustering Network , 2023, AAAI.

[11]  Yue Liu,et al.  Hard Sample Aware Network for Contrastive Deep Graph Clustering , 2022, AAAI.

[12]  Yue Liu,et al.  Contrastive Deep Graph Clustering with Learnable Augmentation , 2022, ArXiv.

[13]  Stan Z. Li,et al.  A Survey of Deep Graph Clustering: Taxonomy, Challenge, and Application , 2022, ArXiv.

[14]  Jiaming Wu,et al.  Embedding Global and Local Influences for Dynamic Graphs , 2022, CIKM.

[15]  Jiliang Tang,et al.  Test-Time Training for Graph Neural Networks , 2022, ArXiv.

[16]  Sihang Zhou,et al.  Attributed Graph Clustering with Dual Redundancy Reduction , 2022, IJCAI.

[17]  Defu Lian,et al.  Cooperative Retriever and Ranker in Deep Recommenders , 2022, WWW.

[18]  Yue Liu,et al.  Mixed Graph Contrastive Network for Semi-Supervised Node Classification , 2022, ACM Transactions on Knowledge Discovery from Data.

[19]  Enhong Chen,et al.  Cache-Augmented Inbatch Importance Resampling for Training Recommender Retriever , 2022, NeurIPS.

[20]  Siwei Wang,et al.  Simple Contrastive Graph Clustering , 2022, IEEE transactions on neural networks and learning systems.

[21]  Min Lin,et al.  Causal Representation Learning for Out-of-Distribution Recommendation , 2022, WWW.

[22]  Enhong Chen,et al.  Learning Recommenders for Implicit Feedback with Importance Resampling , 2022, WWW.

[23]  Yue Liu,et al.  Improved Dual Correlation Reduction Network , 2022, ArXiv.

[24]  En Zhu,et al.  Interpolation-based Contrastive Learning for Few-Label Semi-Supervised Learning , 2022, IEEE transactions on neural networks and learning systems.

[25]  Stan Z. Li,et al.  SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation , 2022, WWW.

[26]  Hao Peng,et al.  Towards Unsupervised Deep Graph Structure Learning , 2022, WWW.

[27]  En Zhu,et al.  Deep Graph Clustering via Dual Correlation Reduction , 2021, AAAI.

[28]  Xinzhong Zhu,et al.  Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance , 2021, IEEE Transactions on Image Processing.

[29]  Chanyoung Park,et al.  Augmentation-Free Self-Supervised Learning on Graphs , 2021, AAAI.

[30]  En Zhu,et al.  Late Fusion Multiple Kernel Clustering With Proxy Graph Refinement , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Siyu Huang,et al.  AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators , 2021, AAAI.

[32]  Enhong Chen,et al.  Fast Variational AutoEncoder with Inverted Multi-Index for Collaborative Filtering , 2021, WWW.

[33]  Wei Jin,et al.  Localized Graph Collaborative Filtering , 2021, SDM.

[34]  Han Zhao,et al.  Graph Debiased Contrastive Learning with Joint Representation Clustering , 2021, IJCAI.

[35]  Meng Liu,et al.  Inductive Representation Learning in Temporal Networks via Mining Neighborhood and Community Influences , 2021, SIGIR.

[36]  Xiaorui Liu,et al.  Automated Self-Supervised Learning for Graphs , 2021, ICLR.

[37]  Zhangyang Wang,et al.  Graph Contrastive Learning Automated , 2021, ICML.

[38]  Jennifer Neville,et al.  Adversarial Graph Augmentation to Improve Graph Contrastive Learning , 2021, NeurIPS.

[39]  Bryan Hooi,et al.  Mixup for Node and Graph Classification , 2021, WWW.

[40]  Jieren Cheng,et al.  Deep Fusion Clustering Network , 2020, AAAI.

[41]  Qiang Liu,et al.  Graph Contrastive Learning with Adaptive Augmentation , 2020, WWW.

[42]  Xiangliang Zhang,et al.  SAIL: Self-Augmented Graph Contrastive Learning , 2020, AAAI.

[43]  Bryan Hooi,et al.  NodeAug: Semi-Supervised Node Classification with Data Augmentation , 2020, KDD.

[44]  Jie Zhou,et al.  Adaptive Graph Encoder for Attributed Graph Embedding , 2020, KDD.

[45]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[46]  Xiao Wang,et al.  Structural Deep Clustering Network , 2020, WWW.

[47]  Noel E. O'Connor,et al.  Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning , 2019, 2020 International Joint Conference on Neural Networks (IJCNN).

[48]  Yun Fu,et al.  Adversarial Graph Embedding for Ensemble Clustering , 2019, IJCAI.

[49]  Jing Jiang,et al.  Attributed Graph Clustering: A Deep Attentional Embedding Approach , 2019, IJCAI.

[50]  Chengqi Zhang,et al.  Learning Graph Embedding With Adversarial Training Methods , 2019, IEEE Transactions on Cybernetics.

[51]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[52]  Qianqian Wang,et al.  Self-consistent Contrastive Attributed Graph Clustering with Pseudo-label Prompt , 2022, IEEE Transactions on Multimedia.

[53]  Yue Liu,et al.  Reasoning over Different Types of Knowledge Graphs: Static, Temporal and Multi-Modal , 2022, ArXiv.

[54]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[55]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..