Graph Adversarial Immunization for Certifiable Robustness

Despite achieving great success, graph neural networks (GNNs) are vulnerable to adversarial attacks. Existing defenses focus on developing adversarial training or robust GNNs. However, little research attention is paid to the potential and practice of immunization on graphs. In this paper, we propose and formulate graph adversarial immunization, i.e., vaccinating part of graph structure to improve certifiable robustness of graph against any admissible adversarial attack. We first propose edge-level immunization to vaccinate node pairs. Despite the primary success, such edge-level immunization cannot defend against emerging node injection attacks, since it only immunizes existing node pairs. To this end, we further propose node-level immunization. To circumvent computationally expensive combinatorial optimization when solving adversarial immunization, we design AdvImmune-Edge and AdvImmune-Node algorithms to effectively obtain the immune node pairs or nodes. Experiments demonstrate the superiority of AdvImmune methods. In particular, AdvImmune-Node remarkably improves the ratio of robust nodes by 79%, 294%, and 100%, after immunizing only 5% nodes. Furthermore, AdvImmune methods show excellent defensive performance against various attacks, outperforming state-of-the-art defenses. To the best of our knowledge, this is the first attempt to improve certifiable robustness from graph data perspective without losing performance on clean graphs, providing new insights into graph adversarial learning.

[1]  Stephan Günnemann,et al.  Collective Robustness Certificates: Exploiting Interdependence in Graph Neural Networks , 2023, ICLR.

[2]  Huawei Shen,et al.  Adversarial Camouflage for Node Injection Attack on Graphs , 2022, ArXiv.

[3]  Xiaoyang Wang,et al.  Graph Neural Network for Fraud Detection via Spatial-Temporal Attention , 2022, IEEE Transactions on Knowledge and Data Engineering.

[4]  Xueqi Cheng,et al.  Single Node Injection Attack against Graph Neural Networks , 2021, CIKM.

[5]  Quan Z. Sheng,et al.  A Comprehensive Survey on Graph Anomaly Detection With Deep Learning , 2021, IEEE Transactions on Knowledge and Data Engineering.

[6]  Evgeny Kharlamov,et al.  TDGIA: Effective Injection Attacks on Graph Neural Networks , 2021, KDD.

[7]  Junzhou Huang,et al.  Adversarial Attack Framework on Graph Embedding Models With Limited Knowledge , 2021, IEEE Transactions on Knowledge and Data Engineering.

[8]  Gavin Taylor,et al.  Robust Optimization as Data Augmentation for Large-scale Graphs , 2020, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zibin Zheng,et al.  Adversarial Attack on Large Scale Graph , 2020, IEEE Transactions on Knowledge and Data Engineering.

[10]  Stephan Günnemann,et al.  Efficient Robustness Certificates for Discrete Data: Sparsity-Aware Randomized Smoothing for Graphs, Images and More , 2020, ICML.

[11]  Jinyuan Jia,et al.  Certified Robustness of Graph Neural Networks against Adversarial Structural Perturbation , 2020, KDD.

[12]  Xiangnan He,et al.  Certifiable Robustness to Discrete Adversarial Perturbations for Factorization Machines , 2020, SIGIR.

[13]  Liang Hou,et al.  Adversarial Immunization for Certifiable Robustness on Graphs , 2020, WSDM.

[14]  Stephan Günnemann,et al.  Certifiable Robustness of Graph Convolutional Networks under Structure Perturbations , 2020, KDD.

[15]  Stephan Günnemann,et al.  Adversarial Attacks on Graph Neural Networks , 2019, GI-Jahrestagung.

[16]  Xiang Zhang,et al.  GNNGuard: Defending Graph Neural Networks against Adversarial Attacks , 2020, NeurIPS.

[17]  Suhang Wang,et al.  Graph Structure Learning for Robust Graph Neural Networks , 2020, KDD.

[18]  Minnan Luo,et al.  Scalable attack on graph data by injecting vicious nodes , 2020, Data Mining and Knowledge Discovery.

[19]  Suhang Wang,et al.  Adversarial Attacks on Graph Neural Networks via Node Injections: A Hierarchical Reinforcement Learning Approach , 2020, WWW.

[20]  Zibin Zheng,et al.  A Survey of Adversarial Learning on Graphs , 2020, ArXiv.

[21]  Jiliang Tang,et al.  Adversarial Attacks and Defenses on Graphs: A Review and Empirical Study , 2020, ArXiv.

[22]  Jinyuan Jia,et al.  Certified Robustness of Community Detection against Adversarial Structural Perturbation via Randomized Smoothing , 2020, WWW.

[23]  Saba A. Al-Sayouri,et al.  All You Need Is Low (Rank): Defending Against Adversarial Attacks on Graphs , 2020, WSDM.

[24]  Stephan Günnemann,et al.  Certifiable Robustness to Graph Perturbations , 2019, NeurIPS.

[25]  Prasenjit Mitra,et al.  Transferring Robustness for Graph Neural Network Against Poisoning Attacks , 2019, WSDM.

[26]  Wenbing Huang,et al.  A Restricted Black-Box Adversarial Framework Towards Attacking Graph Embedding Models , 2019, AAAI.

[27]  Huawei Shen,et al.  Graph Convolutional Networks using Heat Kernel for Semi-supervised Learning , 2019, IJCAI.

[28]  Joey Tianyi Zhou,et al.  Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment , 2019, AAAI.

[29]  Wenwu Zhu,et al.  Robust Graph Convolutional Networks Against Adversarial Attacks , 2019, KDD.

[30]  Rajgopal Kannan,et al.  GraphSAINT: Graph Sampling Based Inductive Learning Method , 2019, ICLR.

[31]  Stephan Gunnemann,et al.  Certifiable Robustness and Robust Training for Graph Convolutional Networks , 2019, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

[32]  Xueqi Cheng,et al.  Popularity Prediction on Social Platforms with Coupled Graph Neural Networks , 2019, WSDM.

[33]  Sijia Liu,et al.  Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective , 2019, IJCAI.

[34]  Karsten M. Borgwardt,et al.  A Persistent Weisfeiler-Lehman Procedure for Graph Classification , 2019, ICML.

[35]  Qiang Li,et al.  Adversarial Training Methods for Network Embedding , 2019, WWW.

[36]  Xueqi Cheng,et al.  Graph Wavelet Neural Network , 2019, ICLR.

[37]  Binghui Wang,et al.  Attacking Graph-based Classification via Manipulating the Graph Structure , 2019, CCS.

[38]  Liming Zhu,et al.  Adversarial Examples on Graph Data: Deep Insights into Attack and Defense , 2019 .

[39]  Stephan Gunnemann,et al.  Adversarial Attacks on Graph Neural Networks via Meta Learning , 2019, ICLR.

[40]  Tat-Seng Chua,et al.  Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure , 2019, IEEE Transactions on Knowledge and Data Engineering.

[41]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[42]  Philip S. Yu,et al.  Adversarial Attack and Defense on Graph Data: A Survey , 2018 .

[43]  Mark Coates,et al.  Bayesian graph convolutional neural networks for semi-supervised classification , 2018, AAAI.

[44]  Stephan Günnemann,et al.  Predict then Propagate: Graph Neural Networks meet Personalized PageRank , 2018, ICLR.

[45]  Stephan Günnemann,et al.  Adversarial Attacks on Node Embeddings via Graph Poisoning , 2018, ICML.

[46]  Junzhou Huang,et al.  Adaptive Sampling Towards Fast Graph Representation Learning , 2018, NeurIPS.

[47]  Yuxiao Dong,et al.  DeepInf: Social Influence Prediction with Deep Learning , 2018, KDD.

[48]  Le Song,et al.  Adversarial Attack on Graph Structured Data , 2018, ICML.

[49]  Stephan Günnemann,et al.  Adversarial Attacks on Neural Networks for Graph Data , 2018, KDD.

[50]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[51]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[52]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[53]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[54]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[55]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[56]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[57]  Andreas Krause,et al.  Cost-effective outbreak detection in networks , 2007, KDD '07.

[58]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[59]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.