Error-Bounded Graph Anomaly Loss for GNNs

Graph neural networks (GNNs) have been widely used to learn node representations from graph data in an unsupervised way for downstream tasks. However, when applied to detect anomalies (e.g., outliers, unexpected density), they deliver unsatisfactory performance as existing loss functions fail. For example, any loss based on random walk (RW) algorithms would no longer work because the assumption that anomalous nodes were close with each other could not hold. Moreover, the nature of class imbalance in anomaly detection tasks brings great challenges to reduce the prediction error. In this work, we propose a novel loss function to train GNNs for anomaly-detectable node representations. It evaluates node similarity using global grouping patterns discovered from graph mining algorithms. It can automatically adjust margins for minority classes based on data distribution. Theoretically, we prove that the prediction error is bounded given the proposed loss function. We empirically investigate the GNN effectiveness of different loss variants based on different algorithms. Experiments on two real-world datasets show that they perform significantly better than RW-based loss for graph anomaly detection.

[1]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[2]  Wenhao Yu,et al.  Identifying Referential Intention with Heterogeneous Contexts , 2020, WWW.

[3]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[4]  Neil Shah,et al.  FLOCK: Combating Astroturfing on Livestreaming Platforms , 2016, WWW.

[5]  Christos Faloutsos,et al.  EigenSpokes: Surprising Patterns and Scalable Community Chipping in Large Graphs , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[6]  Christos Faloutsos,et al.  M-Zoom: Fast Dense-Block Detection in Tensors with Quality Guarantees , 2016, ECML/PKDD.

[7]  Yizhou Sun,et al.  On community outliers and their efficient detection in information networks , 2010, KDD.

[8]  David Mease,et al.  Boosted Classification Trees and Class Probability/Quantile Estimation , 2007, J. Mach. Learn. Res..

[9]  Anna Cinzia Squicciarini,et al.  Combating Crowdsourced Review Manipulators: A Neighborhood-Based Approach , 2018, WSDM.

[10]  Sudipto Guha,et al.  SpotLight: Detecting Anomalies in Streaming Graphs , 2018, KDD.

[11]  Huan Liu,et al.  Deep Anomaly Detection on Attributed Networks , 2019, SDM.

[12]  Christos Faloutsos,et al.  Inferring lockstep behavior from connectivity pattern in large graphs , 2016, Knowledge and Information Systems.

[13]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[14]  Seetha Hari,et al.  Learning From Imbalanced Data , 2019, Advances in Computer and Electrical Engineering.

[15]  Deepayan Chakrabarti,et al.  AutoPart: Parameter-Free Graph Partitioning and Outlier Detection , 2004, PKDD.

[16]  Jimeng Sun,et al.  Neighborhood formation and anomaly detection in bipartite graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[17]  Herna L. Viktor,et al.  Learning from imbalanced data sets with boosting and data generation: the DataBoost-IM approach , 2004, SKDD.

[18]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[19]  Leonardo Neves,et al.  Data Augmentation for Graph Neural Networks , 2020, AAAI.

[20]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[21]  Nitesh V. Chawla,et al.  Calendar Graph Neural Networks for Modeling Time Structures in Spatiotemporal User Behaviors , 2020, KDD.

[22]  Haibo He,et al.  ADASYN: Adaptive synthetic sampling approach for imbalanced learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[23]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[24]  Ming Gao,et al.  BiNE: Bipartite Network Embedding , 2018, SIGIR.

[25]  Chuan Zhou,et al.  Deep Structure Learning for Fraud Detection , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[26]  M. Narasimha Murty,et al.  Outlier Resistant Unsupervised Deep Architectures for Attributed Network Embedding , 2020, WSDM.

[27]  Nitesh V. Chawla,et al.  Heterogeneous Graph Neural Network , 2019, KDD.

[28]  Christos Faloutsos,et al.  BIRDNEST: Bayesian Inference for Ratings-Fraud Detection , 2015, SDM.

[29]  Tong Zhao,et al.  Actionable Objective Optimization for Suspicious Behavior Detection on Large Bipartite Graphs , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[30]  Christos Faloutsos,et al.  Inferring Strange Behavior from Connectivity Pattern in Social Networks , 2014, PAKDD.

[31]  Reid Andersen,et al.  A local algorithm for finding dense subgraphs , 2007, TALG.

[32]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[33]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[34]  V. Koltchinskii,et al.  Empirical margin distributions and bounding the generalization error of combined classifiers , 2002, math/0405343.

[35]  Nitesh V. Chawla,et al.  C4.5 and Imbalanced Data sets: Investigating the eect of sampling method, probabilistic estimate, and decision tree structure , 2003 .

[36]  Christos Faloutsos,et al.  Spotting Suspicious Behaviors in Multimodal Data: A General Metric and Algorithms , 2016, IEEE Transactions on Knowledge and Data Engineering.

[37]  Colin Wei,et al.  Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss , 2019, NeurIPS.

[38]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[39]  Hyun Ah Song,et al.  FRAUDAR: Bounding Graph Fraud in the Face of Camouflage , 2016, KDD.

[40]  Christos Faloutsos,et al.  Edge Weight Prediction in Weighted Signed Networks , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[41]  Christos Faloutsos,et al.  REV2: Fraudulent User Prediction in Rating Platforms , 2018, WSDM.

[42]  Ambuj Tewari,et al.  On the Complexity of Linear Prediction: Risk Bounds, Margin Bounds, and Regularization , 2008, NIPS.

[43]  Christos Faloutsos,et al.  CatchSync: catching synchronized behavior in large directed graphs , 2014, KDD.

[44]  Hyun Ah Song,et al.  Graph-Based Fraud Detection in the Face of Camouflage , 2017, ACM Trans. Knowl. Discov. Data.

[45]  Pasi Fränti,et al.  Outlier Detection Using k-Nearest Neighbour Graph , 2004, ICPR.

[46]  Leman Akoglu,et al.  Collective Opinion Spam Detection: Bridging Review Networks and Metadata , 2015, KDD.

[47]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[48]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[49]  Hongxia Yang,et al.  Hierarchical Representation Learning for Bipartite Graphs , 2019, IJCAI.

[50]  Christos Faloutsos,et al.  Spotting Suspicious Link Behavior with fBox: An Adversarial Perspective , 2014, 2014 IEEE International Conference on Data Mining.

[51]  Ling Chen,et al.  Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection , 2018, KDD.

[52]  Huan Liu,et al.  Graph Neural Networks with High-order Feature Interactions , 2019, ArXiv.

[53]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[54]  Meng Jiang Catching Social Media Advertisers with Strategy Analysis , 2016, CyberSafety@CIKM.

[55]  Hisao Tamaki,et al.  Greedily Finding a Dense Subgraph , 2000, J. Algorithms.

[56]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[57]  Christos Faloutsos,et al.  oddball: Spotting Anomalies in Weighted Graphs , 2010, PAKDD.

[58]  Christos Faloutsos,et al.  Opinion Fraud Detection in Online Reviews by Network Effects , 2013, ICWSM.