G2uardFL: Safeguarding Federated Learning Against Backdoor Attacks through Attributed Client Graph Clustering

As a collaborative paradigm, Federated Learning (FL) empowers clients to engage in collective model training without exchanging their respective local data. Nevertheless, FL remains vulnerable to backdoor attacks in which an attacker compromises malicious clients, and injects poisoned model weights into the aggregation process to yield attacker-chosen predictions for particular samples. Existing countermeasures, mainly based on anomaly detection, may erroneously reject legitimate weights while accepting malicious ones, which is due to inadequacies in quantifying client model similarities. Other defense mechanisms prove effective exclusively when confronted with a restricted number of malicious clients, e.g., less than 10%. To address these vulnerabilities, we present G$^2$uardFL, a protective framework that reframes the detection of malicious clients as an attributed graph clustering problem, thereby safeguarding FL systems. This framework employs a client graph clustering technique to identify malicious clients and incorporates an adaptive method to amplify the disparity between the aggregated model and poisoned client models, thereby eliminating previously embedded backdoors. A theoretical analysis of convergence is also performed to demonstrate that the global model closely approximates the model untouched by any backdoor. Through empirical evaluation compared to cutting-edge defenses and against various backdoor attacks, our experimental results indicate that G$^2$uardFL considerably undermines the effectiveness of backdoor attacks while maintaining a negligible impact on the benign sample performance.

[1]  Pengyuan Zhou,et al.  Mitigating Backdoors in Federated Learning with FLD , 2023, ArXiv.

[2]  Pin-Yu Chen,et al.  FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning , 2022, ICLR.

[3]  Xiaoyu Cao,et al.  FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clients , 2022, KDD.

[4]  Xin Liu,et al.  Cross-Silo Federated Learning: Challenges and Opportunities , 2022, ArXiv.

[5]  J. Zhang,et al.  Layer-wised Model Aggregation for Personalized Federated Learning , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ahmad-Reza Sadeghi,et al.  DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection , 2022, NDSS.

[7]  Amin Hassanzadeh,et al.  FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective , 2021, NeurIPS.

[8]  Tao Xiang,et al.  Z-Score Normalization, Hubness, and Few-Shot Learning , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[9]  Riadh Ksantini,et al.  Rethinking Graph Auto-Encoder Models for Attributed Graph Clustering , 2021, IEEE Transactions on Knowledge and Data Engineering.

[10]  Minghao Chen,et al.  CRFL: Certifiably Robust Federated Learning against Backdoor Attacks , 2021, ICML.

[11]  Azalia Mirhoseini,et al.  FLAME: Taming Backdoors in Federated Learning (Extended Version 1) , 2021, 2101.02281.

[12]  Xiaoyu Cao,et al.  FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping , 2020, NDSS.

[13]  Shiva Raj Pokhrel Federated learning meets blockchain at 6G edge: a drone-assisted networking for disaster response , 2020, DroneCom@MOBICOM.

[14]  H. Poor,et al.  When Federated Learning Meets Blockchain: A New Distributed Learning Paradigm , 2020, IEEE Computational Intelligence Magazine.

[15]  Kartik Sreenivasan,et al.  Attack of the Tails: Yes, You Really Can Backdoor Federated Learning , 2020, NeurIPS.

[16]  Yulia R. Gel,et al.  Defending Against Backdoors in Federated Learning with Robust Learning Rate , 2020, AAAI.

[17]  Ben Y. Zhao,et al.  Backdoor Attacks Against Deep Learning Systems in the Physical World , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  M. Bennis,et al.  Federated Learning in Vehicular Networks , 2020, 2022 IEEE International Mediterranean Conference on Communications and Networking (MeditCom).

[19]  Thomas Wiegand,et al.  On the Byzantine Robustness of Clustered Federated Learning , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Bo Li,et al.  DBA: Distributed Backdoor Attacks against Federated Learning , 2020, ICLR.

[21]  Qinghua Hu,et al.  Collaborative Graph Convolutional Networks: Unsupervised Learning Meets Semi-Supervised Learning , 2020, AAAI.

[22]  Martin Jaggi,et al.  A Unified Theory of Decentralized SGD with Changing Topology and Local Updates , 2020, ICML.

[23]  Zaïd Harchaoui,et al.  Robust Aggregation for Federated Learning , 2019, IEEE Transactions on Signal Processing.

[24]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[25]  Di Cao,et al.  Understanding Distributed Poisoning Attack in Federated Learning , 2019, 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS).

[26]  Ananda Theertha Suresh,et al.  Can You Really Backdoor Federated Learning? , 2019, ArXiv.

[27]  Tony Q. S. Quek,et al.  On Safeguarding Privacy and Security in the Framework of Federated Learning , 2019, IEEE Network.

[28]  Tzu-Ming Harry Hsu,et al.  Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification , 2019, ArXiv.

[29]  Xiang Li,et al.  On the Convergence of FedAvg on Non-IID Data , 2019, ICLR.

[30]  Baharan Mirzasoleiman,et al.  Coresets for Data-efficient Training of Machine Learning Models , 2019, ICML.

[31]  Håkan Grahn,et al.  ARDIS: a Swedish historical handwritten digit dataset , 2019, Neural Computing and Applications.

[32]  Klaus-Robert Müller,et al.  Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Prateek Mittal,et al.  Analyzing Federated Learning through an Adversarial Lens , 2018, ICML.

[34]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[35]  Michael G. Rabbat,et al.  Stochastic Gradient Push for Distributed Deep Learning , 2018, ICML.

[36]  Ivan Beschastnikh,et al.  Mitigating Sybils in Federated Learning Poisoning , 2018, ArXiv.

[37]  Vitaly Shmatikov,et al.  How To Backdoor Federated Learning , 2018, AISTATS.

[38]  Kannan Ramchandran,et al.  Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates , 2018, ICML.

[39]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[40]  Rachid Guerraoui,et al.  Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent , 2017, NIPS.

[41]  Wei Zhang,et al.  Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent , 2017, NIPS.

[42]  Manish Singh,et al.  Efficient Twitter sentiment classification using subjective distant supervision , 2017, 2017 9th International Conference on Communication Systems and Networks (COMSNETS).

[43]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[44]  Zhen Li,et al.  Towards Better Analysis of Deep Convolutional Neural Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.

[45]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[48]  Ricardo J. G. B. Campello,et al.  Density-Based Clustering Based on Hierarchical Density Estimates , 2013, PAKDD.

[49]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[50]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[51]  Robert Reams,et al.  Hadamard inverses, square roots and products of almost semidefinite matrices , 1999 .

[52]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[53]  Mohamed-Slim Alouini,et al.  FilFL: Accelerating Federated Learning via Client Filtering , 2023, ArXiv.

[54]  Ivan Beschastnikh,et al.  The Limitations of Federated Learning in Sybil Settings , 2020, RAID.

[55]  Markus Miettinen,et al.  Poisoning Attacks on Federated Learning-based IoT Intrusion Detection System , 2020, Proceedings 2020 Workshop on Decentralized IoT Systems and Security.

[56]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[57]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.