Towards On-Device Federated Learning: A Direct Acyclic Graph-based Blockchain Approach

Due to the distributed characteristics of federated learning (FL), the vulnerability of the global model and the coordination of devices are the main obstacle. As a promising solution of decentralization, scalability, and security, leveraging the blockchain in FL has attracted much attention in recent years. However, the traditional consensus mechanisms designed for blockchain-like proof of work (PoW) would cause extreme resource consumption, which reduces the efficiency of FL greatly, especially when the participating devices are wireless and resource-limited. In order to address device asynchrony and anomaly detection in FL while avoiding the extra resource consumption caused by blockchain, this article introduces a framework for empowering FL using direct acyclic graph (DAG)-based blockchain systematically (DAG-FL). Accordingly, DAG-FL is first introduced from a three-layer architecture in detail, and then, two algorithms DAG-FL Controlling and DAG-FL Updating are designed running on different nodes to elaborate the operation of the DAG-FL consensus mechanism. After that, a Poisson process model is formulated to discuss that how to set deployment parameters to maintain DAG-FL stably in different FL tasks. The extensive simulations and experiments show that DAG-FL can achieve better performance in terms of training efficiency and model accuracy compared with the typical existing on-device FL systems as the benchmarks.

[1]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[2]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  Khaled Salah,et al.  Blockchain for AI: Review and Open Research Challenges , 2019, IEEE Access.

[4]  Mugen Peng,et al.  Performance analysis and comparison of PoW, PoS and DAG based blockchains , 2020, Digit. Commun. Networks.

[5]  Shahid Mumtaz,et al.  When Internet of Things Meets Blockchain: Challenges in Distributed Consensus , 2019, IEEE Network.

[6]  Tooska Dargahi,et al.  A systematic , 2022 .

[7]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[8]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[9]  Long Zhang,et al.  Direct Acyclic Graph based Blockchain for Internet of Things: Performance and Security Analysis , 2019, ArXiv.

[10]  Vitaly Shmatikov,et al.  How To Backdoor Federated Learning , 2018, AISTATS.

[11]  Seong-Lyun Kim,et al.  Blockchained On-Device Federated Learning , 2018, IEEE Communications Letters.

[12]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[13]  Hao Xu,et al.  RAFT Based Wireless Blockchain Networks in the Presence of Malicious Jamming , 2020, IEEE Wireless Communications Letters.

[14]  Yan Zhang,et al.  Blockchain and Federated Learning for Privacy-Preserved Data Sharing in Industrial IoT , 2020, IEEE Transactions on Industrial Informatics.

[15]  Zibin Zheng,et al.  PiRATE: A Blockchain-Based Secure Framework of Distributed Machine Learning in 5G Networks , 2019, IEEE Network.

[16]  Satoshi Nakamoto.  Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[17]  Ke Zhang,et al.  Blockchain Empowered Asynchronous Federated Learning for Secure Data Sharing in Internet of Vehicles , 2020, IEEE Transactions on Vehicular Technology.

[18]  Tao Jiang,et al.  Pervasive intelligent endogenous 6G wireless systems: Prospects, theories and key technologies , 2020, Digit. Commun. Networks.

[19]  Ben Y. Zhao,et al.  Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[20]  Adi Shamir,et al.  A method for obtaining digital signatures and public-key cryptosystems , 1978, CACM.

[21]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[22]  Indranil Gupta,et al.  SLSGD: Secure and Efficient Distributed On-device Machine Learning , 2019, ECML/PKDD.

[23]  Indranil Gupta,et al.  Asynchronous Federated Optimization , 2019, ArXiv.

[24]  Stephen A. Jarvis,et al.  SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead , 2019, IEEE Transactions on Computers.

[25]  Choong Seon Hong,et al.  FLchain: Federated Learning via MEC-enabled Blockchain Network , 2019, 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[26]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[27]  Huzefa Rangwala,et al.  Asynchronous Online Federated Learning for Edge Devices , 2019, ArXiv.

[28]  Khaled Salah,et al.  Towards Blockchain-Based Reputation-Aware Federated Learning , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[29]  Yue Zhang,et al.  DeepChain: Auditable and Privacy-Preserving Deep Learning with Blockchain-Based Incentive , 2019, IEEE Transactions on Dependable and Secure Computing.

[30]  Wei Zhang,et al.  Asynchronous Decentralized Parallel Stochastic Gradient Descent , 2017, ICML.

[31]  Marco Scavuzzo,et al.  Asynchronous Federated Learning for Geospatial Applications , 2018, DMLE/IOTSTREAMING@PKDD/ECML.

[32]  Bing Chen,et al.  Poisoning Attack in Federated Learning using Generative Adversarial Nets , 2019, 2019 18th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/13th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE).

[33]  Tianjian Chen,et al.  Abnormal Client Behavior Detection in Federated Learning , 2019, ArXiv.

[34]  S. Popov.  The Tangle , 2015 .

[35]  Choong Seon Hong,et al.  Blockchain-based Node-aware Dynamic Weighting Methods for Improving Federated Learning Performance , 2019, 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[36]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.