Transferring Robustness for Graph Neural Network Against Poisoning Attacks

Graph neural networks (GNNs) are widely used in many applications. However, their robustness against adversarial attacks is criticized. Prior studies show that using unnoticeable modifications on graph topology or nodal features can significantly reduce the performances of GNNs. It is very challenging to design robust graph neural networks against poisoning attack and several efforts have been taken. Existing work aims at reducing the negative impact from adversarial edges only with the poisoned graph, which is sub-optimal since they fail to discriminate adversarial edges from normal ones. On the other hand, clean graphs from similar domains as the target poisoned graph are usually available in the real world. By perturbing these clean graphs, we create supervised knowledge to train the ability to detect adversarial edges so that the robustness of GNNs is elevated. However, such potential for clean graphs is neglected by existing work. To this end, we investigate a novel problem of improving the robustness of GNNs against poisoning attacks by exploring clean graphs. Specifically, we propose PA-GNN, which relies on a penalized aggregation mechanism that directly restrict the negative impact of adversarial edges by assigning them lower attention coefficients. To optimize PA-GNN for a poisoned graph, we design a meta-optimization algorithm that trains PA-GNN to penalize perturbations using clean graphs and their adversarial counterparts, and transfers such ability to improve the robustness of PA-GNN on the poisoned graph. Experimental results on four real-world datasets demonstrate the robustness of PA-GNN against poisoning attacks on graphs.

[1]  Xianfeng Tang,et al.  Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction , 2019, WWW.

[2]  Liming Zhu,et al.  Adversarial Examples on Graph Data: Deep Insights into Attack and Defense , 2019 .

[3]  Le Song,et al.  Adversarial Attack on Graph Structured Data , 2018, ICML.

[4]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[5]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[6]  Charu C. Aggarwal,et al.  Graph Convolutional Networks with EigenPooling , 2019, KDD.

[7]  Wenwu Zhu,et al.  Robust Graph Convolutional Networks Against Adversarial Attacks , 2019, KDD.

[8]  Stephan Günnemann,et al.  Adversarial Attacks on Neural Networks for Graph Data , 2018, KDD.

[9]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[10]  Zhengyang Wang,et al.  Large-Scale Learnable Graph Convolutional Networks , 2018, KDD.

[11]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[12]  Stephan Gunnemann,et al.  Certifiable Robustness and Robust Training for Graph Convolutional Networks , 2019, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

[13]  Jinfeng Yi,et al.  ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models , 2017, AISec@CCS.

[14]  Suhang Wang,et al.  Attacking Graph Convolutional Networks via Rewiring , 2019, ArXiv.

[15]  Ying Wei,et al.  Hierarchically Structured Meta-learning , 2019, ICML.

[16]  Hong Yu,et al.  Meta Networks , 2017, ICML.

[17]  Hao Ma,et al.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs , 2018, UAI.

[18]  Jiliang Tang,et al.  Adversarial Attacks and Defenses in Images, Graphs and Text: A Review , 2019, International Journal of Automation and Computing.

[19]  Nitesh V. Chawla,et al.  Graph Few-shot Learning via Knowledge Transfer , 2020, AAAI.

[20]  Li Zhang,et al.  Learning to Learn: Meta-Critic Networks for Sample Efficient Learning , 2017, ArXiv.

[21]  Nitesh V. Chawla,et al.  Online Purchase Prediction via Multi-Scale Modeling of Behavior Dynamics , 2019, KDD.

[22]  Ruoyu Li,et al.  Adaptive Graph Convolutional Neural Networks , 2018, AAAI.

[23]  Zhiyuan Liu,et al.  Graph Neural Networks with Generated Parameters for Relation Extraction , 2019, ACL.

[24]  Ming Jin,et al.  Power up! Robust Graph Convolutional Network against Evasion Attacks based on Graph Powering , 2019, ArXiv.

[25]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[26]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[27]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[28]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[29]  Huan Liu,et al.  Interactive Anomaly Detection on Attributed Networks , 2019, WSDM.

[30]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[31]  Sijia Liu,et al.  Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective , 2019, IJCAI.

[32]  Song Bai,et al.  Learning Transferable Adversarial Examples via Ghost Networks , 2018, AAAI.

[33]  Charu C. Aggarwal,et al.  Multi-dimensional Graph Convolutional Networks , 2018, SDM.

[34]  Huan Liu,et al.  Graph Neural Networks with High-order Feature Interactions , 2019, ArXiv.

[35]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Wei Wang,et al.  Click Feedback-Aware Query Recommendation Using Adversarial Examples , 2019, WWW.

[37]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[38]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Stephan Günnemann,et al.  Adversarial Attacks on Node Embeddings via Graph Poisoning , 2018, ICML.

[40]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[41]  Sepp Hochreiter,et al.  Learning to Learn Using Gradient Descent , 2001, ICANN.

[42]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[43]  Stephan Gunnemann,et al.  Adversarial Attacks on Graph Neural Networks via Meta Learning , 2019, ICLR.

[44]  Lise Getoor,et al.  Collective Classi!cation in Network Data , 2008 .

[45]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[46]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[47]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[48]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[49]  Yuanbin Wu,et al.  Graph-based Dependency Parsing with Graph Neural Networks , 2019, ACL.

[50]  Song Bai,et al.  Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses , 2019, ECCV.

[51]  Jinfeng Yi,et al.  Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach , 2018, ICLR.

[52]  Qi Xuan,et al.  Fast Gradient Attack on Network Embedding , 2018, ArXiv.

[53]  Tong Zhang,et al.  NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks , 2019, ICML.

[54]  Huan Liu,et al.  CrossFire: Cross Media Joint Friend and Item Recommendations , 2018, WSDM.

[55]  Vasant Honavar,et al.  Node Injection Attacks on Graphs via Reinforcement Learning , 2019, ArXiv.

[56]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[57]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[58]  Sungroh Yoon,et al.  Transfer Learning for Deep Learning on Graph-Structured Data , 2016, AAAI.

[59]  Huan Liu,et al.  Deep Anomaly Detection on Attributed Networks , 2019, SDM.