Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning

Anomaly detection on attributed networks attracts considerable research interests due to wide applications of attributed networks in modeling a wide range of complex systems. Recently, the deep learning-based anomaly detection methods have shown promising results over shallow approaches, especially on networks with high-dimensional attributes and complex structures. However, existing approaches, which employ graph autoencoder as their backbone, do not fully exploit the rich information of the network, resulting in suboptimal performance. Furthermore, these methods do not directly target anomaly detection in their learning objective and fail to scale to large networks due to the full graph training mechanism. To overcome these limitations, in this article, we present a novel Contrastive self-supervised Learning framework for Anomaly detection on attributed networks (CoLA for abbreviation). Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair, which can capture the relationship between each node and its neighboring substructure in an unsupervised way. Meanwhile, a well-designed graph neural network (GNN)-based contrastive learning model is proposed to learn informative embedding from high-dimensional attributes and local structure and measure the agreement of each instance pairs with its outputted scores. The multiround predicted scores by the contrastive learning model are further used to evaluate the abnormality of each node with statistical estimation. In this way, the learning model is trained by a specific anomaly detection-aware target. Furthermore, since the input of the GNN module is batches of instance pairs instead of the full network, our framework can adapt to large networks flexibly. Experimental results show that our proposed framework outperforms the state-of-the-art baseline methods on all seven benchmark data sets.

[1]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[2]  Minnan Luo,et al.  Graph Representation Learning via Graphical Mutual Information Maximization , 2020, WWW.

[3]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[4]  Longbing Cao,et al.  Deep Learning for Anomaly Detection: A Review , 2020, ArXiv.

[5]  David B. Skillicorn,et al.  Detecting Anomalies in Graphs , 2007, 2007 IEEE Intelligence and Security Informatics.

[6]  Le Song,et al.  Heterogeneous Graph Neural Networks for Malicious Account Detection , 2018, CIKM.

[7]  Lingfan Yu,et al.  Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks. , 2019 .

[8]  Xiaowei Xu,et al.  SCAN: a structural clustering algorithm for networks , 2007, KDD '07.

[9]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[10]  Chengqi Zhang,et al.  Learning Graph Embedding With Adversarial Training Methods , 2019, IEEE Transactions on Cybernetics.

[11]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Xin Li,et al.  Your Style Your Identity: Leveraging Writing and Photography Styles for Drug Trafficker Identification in Darknet Markets over Attributed Heterogeneous Information Network , 2019, WWW.

[13]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[14]  Jundong Li,et al.  SpecAE: Spectral AutoEncoder for Anomaly Detection in Attributed Networks , 2019, CIKM.

[15]  Stephan Günnemann,et al.  Diffusion Improves Graph Learning , 2019, NeurIPS.

[16]  J. Leskovec,et al.  Open Graph Benchmark: Datasets for Machine Learning on Graphs , 2020, NeurIPS.

[17]  Leman Akoglu,et al.  Scalable Anomaly Ranking of Attributed Neighborhoods , 2016, SDM.

[18]  Yixin Chen,et al.  Weisfeiler-Lehman Neural Machine for Link Prediction , 2017, KDD.

[19]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[20]  Charu C. Aggarwal,et al.  NetWalk: A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks , 2018, KDD.

[21]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[22]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Chen Gong,et al.  Hyperspectral Image Classification With Context-Aware Dynamic Graph Convolutional Network , 2019, IEEE Trans. Geosci. Remote. Sens..

[24]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[25]  Michal Valko,et al.  Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning , 2020, NeurIPS.

[26]  Jie Tang,et al.  Self-Supervised Learning: Generative or Contrastive , 2020, IEEE Transactions on Knowledge and Data Engineering.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Huan Liu,et al.  Deep Anomaly Detection on Attributed Networks , 2019, SDM.

[29]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[30]  Xinlei Chen,et al.  Exploring Simple Siamese Representation Learning , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[32]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[33]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[34]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[35]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[36]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[37]  Huan Liu,et al.  Interactive Anomaly Detection on Attributed Networks , 2019, WSDM.

[38]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[39]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[40]  Anton van den Hengel,et al.  Deep Anomaly Detection with Deviation Networks , 2019, KDD.

[41]  Yongfeng Zhang,et al.  Reinforcement Knowledge Graph Reasoning for Explainable Recommendation , 2019, SIGIR.

[42]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[43]  Yoshua Bengio,et al.  Learning deep representations by mutual information estimation and maximization , 2018, ICLR.

[44]  Sanjay Ranka,et al.  Conditional Anomaly Detection , 2007, IEEE Transactions on Knowledge and Data Engineering.

[45]  Rajgopal Kannan,et al.  GraphSAINT: Graph Sampling Based Inductive Learning Method , 2019, ICLR.

[46]  Huan Liu,et al.  Radar: Residual Analysis for Anomaly Detection in Attributed Networks , 2017, IJCAI.

[47]  Bo Zong,et al.  Self-Attentive Attributed Network Embedding Through Adversarial Learning , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[48]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[49]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[50]  Christos Faloutsos,et al.  Sampling from large graphs , 2006, KDD '06.

[51]  Qinghua Zheng,et al.  ANOMALOUS: A Joint Modeling Approach for Anomaly Detection on Attributed Networks , 2018, IJCAI.

[52]  Jing Jiang,et al.  Graph WaveNet for Deep Spatial-Temporal Graph Modeling , 2019, IJCAI.

[53]  Yuxiao Dong,et al.  GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training , 2020, KDD.

[54]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[55]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[56]  Takanori Maehara,et al.  Revisiting Graph Neural Networks: All We Have is Low-Pass Filters , 2019, ArXiv.

[57]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[58]  Xiaojun Chang,et al.  Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks , 2020, KDD.

[59]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[60]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[61]  Xiangnan He,et al.  Attributed Social Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[62]  Gholamreza Haffari,et al.  Reasoning Like Human: Hierarchical Reinforcement Learning for Knowledge Graph Reasoning , 2020, IJCAI.

[63]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[64]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[65]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[66]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[67]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[68]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[69]  Christos Faloutsos,et al.  Fast Random Walk with Restart and Its Applications , 2006, Sixth International Conference on Data Mining (ICDM'06).

[70]  Mengxiao Zhu,et al.  MixedAD: A Scalable Algorithm for Detecting Mixed Anomalies in Attributed Graphs , 2020, AAAI.