Anomaly Detection in Dynamic Graphs via Transformer

Detecting anomalies for dynamic graphs has drawn increasing attention due to their wide applications in social networks, e-commerce, and cybersecurity. Recent deep learning-based approaches have shown promising results over shallow methods. However, they fail to address two core challenges of anomaly detection in dynamic graphs: the lack of informative encoding for unattributed nodes and the difficulty of learning discriminate knowledge from coupled spatial-temporal dynamic graphs. To overcome these challenges, in this paper, we present a novel Transformer-based Anomaly Detection framework for DYnamic graphs (TADDY). Our framework constructs a comprehensive node encoding strategy to better represent each node’s structural and temporal roles in an evolving graphs stream. Meanwhile, TADDY captures informative representation from dynamic graphs with coupled spatial-temporal patterns via a dynamic graph transformer model. The extensive experimental results demonstrate that our proposed TADDY framework outperforms the state-of-the-art methods by a large margin on six real-world datasets.

[1]  Haifeng Chen,et al.  Structural Temporal Graph Neural Networks for Anomaly Detection in Dynamic Graphs , 2020, CIKM.

[2]  Yue Wu,et al.  H-VGRAE: A Hierarchical Stochastic Spatial-Temporal Embedding Method for Robust Anomaly Detection in Dynamic Networks , 2020, ArXiv.

[3]  Yi Tay,et al.  Efficient Transformers: A Survey , 2020, ArXiv.

[4]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[5]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[6]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[7]  Jiawei Zhang,et al.  Graph-Bert: Only Attention is Needed for Learning Graph Representations , 2020, ArXiv.

[8]  Zhiwen Yu,et al.  Influence Spread in Geo-Social Networks: A Multiobjective Optimization Perspective , 2019, IEEE Transactions on Cybernetics.

[9]  Leman Akoglu,et al.  Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs , 2016, KDD.

[10]  Risi Kondor,et al.  Diffusion kernels on graphs and other discrete structures , 2002, ICML 2002.

[11]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[12]  Daniel Massey,et al.  Collecting the internet AS-level topology , 2005, CCRV.

[13]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[14]  D. Tao,et al.  A Survey on Visual Transformer , 2020, ArXiv.

[15]  Steve Harenberg,et al.  A Scalable Approach for Outlier Detection in Edge Streams Using Sketch-based Approximations , 2016, SDM.

[16]  Liangtian Wan,et al.  Graph Learning: A Survey , 2021, IEEE Transactions on Artificial Intelligence.

[17]  Shirui Pan,et al.  Temporal Network Embedding for Link Prediction via VAE Joint Attention Mechanism , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Jun Gao,et al.  Category-aware Graph Neural Networks for Improving E-commerce Review Helpfulness Prediction , 2020, CIKM.

[19]  Yatao Bian,et al.  Self-Supervised Graph Transformer on Large-Scale Molecular Data , 2020, NeurIPS.

[20]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[21]  Philip S. Yu,et al.  A Survey on Knowledge Graphs: Representation, Acquisition, and Applications , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Kumar Sricharan,et al.  Localizing anomalous changes in time-evolving graphs , 2014, SIGMOD Conference.

[23]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[24]  Munmun De Choudhury,et al.  Social Synchrony: Predicting Mimicry of User Actions in Online Social Media , 2009, 2009 International Conference on Computational Science and Engineering.

[25]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[26]  Stephan Günnemann,et al.  Diffusion Improves Graph Learning , 2019, NeurIPS.

[27]  Jian Li,et al.  AddGraph: Anomaly Detection in Dynamic Graph Using Attention-based Temporal GCN , 2019, IJCAI.

[28]  Yizhou Sun,et al.  Heterogeneous Graph Transformer , 2020, WWW.

[29]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[30]  Philip S. Yu,et al.  Outlier detection in graph streams , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[31]  Philip S. Yu,et al.  HinCTI: A Cyber Threat Intelligence Modeling and Identification System Based on Heterogeneous Information Network , 2020, IEEE Transactions on Knowledge and Data Engineering.

[32]  Tao Xiang,et al.  Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Charu C. Aggarwal,et al.  NetWalk: A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks , 2018, KDD.

[34]  Georg Heigold,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2021, ICLR.

[35]  Jaewoo Kang,et al.  Graph Transformer Networks , 2019, NeurIPS.

[36]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[37]  Ryan A. Rossi,et al.  The Network Data Repository with Interactive Graph Analytics and Visualization , 2015, AAAI.

[38]  Christos Faloutsos,et al.  Edge Weight Prediction in Weighted Signed Networks , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[39]  Christos Faloutsos,et al.  REV2: Fraudulent User Prediction in Rating Platforms , 2018, WSDM.

[40]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[41]  Tore Opsahl,et al.  Clustering in weighted networks , 2009, Soc. Networks.

[42]  Ting Guo,et al.  Clustering social audiences in business information networks , 2020, Pattern Recognit..

[43]  Chuan Zhou,et al.  Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning , 2021, IEEE transactions on neural networks and learning systems.

[44]  Philip S. Yu,et al.  Lime: Low-Cost and Incremental Learning for Dynamic Heterogeneous Information Networks , 2021, IEEE Transactions on Computers.

[45]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[46]  Philip S. Yu,et al.  A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning , 2021, IEEE Transactions on Knowledge and Data Engineering.

[47]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[48]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[49]  Philip S. Yu,et al.  Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks , 2021, ACM Trans. Knowl. Discov. Data.

[50]  Sudipto Guha,et al.  SpotLight: Detecting Anomalies in Streaming Graphs , 2018, KDD.

[51]  Lu Liu,et al.  A Universal Representation Transformer Layer for Few-Shot Image Classification , 2020, ICLR.

[52]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.