Deep Graph-level Anomaly Detection by Glocal Knowledge Distillation

Graph-level anomaly detection (GAD) describes the problem of detecting graphs that are abnormal in their structure and/or the features of their nodes, as compared to other graphs. One of the challenges in GAD is to devise graph representations that enable the detection of both locallyand globally-anomalous graphs, i.e., graphs that are abnormal in their fine-grained (node-level) or holistic (graph-level) properties, respectively. To tackle this challenge we introduce a novel deep anomaly detection approach for GAD that learns rich global and local normal pattern information by joint random distillation of graph and node representations. The random distillation is achieved by training one GNN to predict another GNN with randomly initialized network weights. Extensive experiments on 16 real-world graph datasets from diverse domains show that our model significantly outperforms seven state-of-the-art models. Code and datasets are available at https://git.io/GLocalKD. CCS CONCEPTS •Computingmethodologies→ Semi-supervised learning settings; Neural networks; Anomaly detection.

[1]  Kumar Sricharan,et al.  Localizing anomalous changes in time-evolving graphs , 2014, SIGMOD Conference.

[2]  Hamid R. Rabiee,et al.  Multiresolution Knowledge Distillation for Anomaly Detection , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Fahad Shahbaz Khan,et al.  Anomaly Detection in Video via Self-Supervised and Multi-Task Learning , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Hamida Seba,et al.  A simple graph embedding for anomaly detection in a stream of heterogeneous labeled graphs , 2020, Pattern Recognit..

[5]  Jian Tang,et al.  InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization , 2019, ICLR.

[6]  Jing Wang,et al.  Unsupervised Anomaly Detection with Distillated Teacher-Student Network Ensemble , 2021, Entropy.

[7]  Shiqi Wang,et al.  Face Anti-Spoofing With Deep Neural Network Distillation , 2020, IEEE Journal of Selected Topics in Signal Processing.

[8]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[9]  Jun Li,et al.  One-Class Adversarial Nets for Fraud Detection , 2018, AAAI.

[10]  Jian Li,et al.  AddGraph: Anomaly Detection in Dynamic Graph Using Attention-based Temporal GCN , 2019, IJCAI.

[11]  Leman Akoglu,et al.  On Using Classification Datasets to Evaluate Graph Outlier Detection: Peculiar Observations and New Insights , 2021, Big Data.

[12]  Leman Akoglu,et al.  Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs , 2016, KDD.

[13]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[14]  Atsutoshi Kumagai,et al.  Semi-supervised Anomaly Detection on Attributed Graphs , 2020, 2021 International Joint Conference on Neural Networks (IJCNN).

[15]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Georg Langs,et al.  f‐AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks , 2019, Medical Image Anal..

[17]  Hwee Kuan Lee,et al.  Fence GAN: Towards Better Anomaly Detection , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[18]  Chao Liu,et al.  Anomaly Detection with Graph Convolutional Networks for Insider Threat and Fraud Detection , 2019, MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM).

[19]  S. Hewitt,et al.  2007 , 2018, Los 25 años de la OMC: Una retrospectiva fotográfica.

[20]  Hongxing He,et al.  Outlier Detection Using Replicator Neural Networks , 2002, DaWaK.

[21]  Leman Akoglu,et al.  Anomaly Detection in Large Labeled Multi-Graph Databases , 2020, ArXiv.

[22]  Yang Liu,et al.  graph2vec: Learning Distributed Representations of Graphs , 2017, ArXiv.

[23]  Charu C. Aggarwal,et al.  NetWalk: A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks , 2018, KDD.

[24]  Chun Yen Lee,et al.  Descriptive prediction of drug side‐effects using a hybrid deep learning model , 2021, Int. J. Intell. Syst..

[25]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[26]  Baihong Jin,et al.  One-Class Graph Neural Networks for Anomaly Detection in Attributed Networks , 2020 .

[27]  Tianwen Jiang,et al.  Error-Bounded Graph Anomaly Loss for GNNs , 2020, CIKM.

[28]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[29]  Christos Faloutsos,et al.  Fast and Accurate Anomaly Detection in Dynamic Graphs with a Two-Pronged Approach , 2019, KDD.

[30]  Huan Liu,et al.  Inductive Anomaly Detection on Attributed Networks , 2020, IJCAI.

[31]  Ling Chen,et al.  Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection , 2018, KDD.

[32]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[33]  Anton van den Hengel,et al.  Deep Anomaly Detection with Deviation Networks , 2019, KDD.

[34]  Amos J. Storkey,et al.  Exploration by Random Network Distillation , 2018, ICLR.

[35]  Roman Garnett,et al.  Propagation kernels: efficient graph kernels from propagated information , 2015, Machine Learning.

[36]  Markus Neuhäuser,et al.  Wilcoxon Signed Rank Test , 2006 .

[37]  Kai Ming Ting,et al.  LeSiNN: Detecting Anomalies by Identifying Least Similar Nearest Neighbours , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[38]  Albin Cassirer,et al.  Randomized Prior Functions for Deep Reinforcement Learning , 2018, NeurIPS.

[39]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[40]  Sudipto Guha,et al.  SpotLight: Detecting Anomalies in Streaming Graphs , 2018, KDD.

[41]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[42]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[43]  C. Steger,et al.  Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[45]  Chunhua Shen,et al.  Unsupervised Representation Learning by Predicting Random Distances , 2019, IJCAI.

[46]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[47]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[48]  Vishal M. Patel,et al.  Learning Deep Features for One-Class Classification , 2018, IEEE Transactions on Image Processing.

[49]  Arthur Zimek,et al.  On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study , 2016, Data Mining and Knowledge Discovery.

[50]  Charu C. Aggarwal,et al.  Graph Data Management and Mining: A Survey of Algorithms and Applications , 2010, Managing and Mining Graph Data.

[51]  Charu C. Aggarwal,et al.  Outlier Detection with Autoencoder Ensembles , 2017, SDM.

[52]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.