Communication-Efficient Federated Learning for Digital Twin Edge Networks in Industrial IoT

The rapid development of Artificial Intelligence (AI) and 5G paradigm, opens up new possibilities for emerging applications in Industrial Internet of Things (IIoT). However, the large amount of data, the limited resources of IoT devices, and the increasing concerns of data privacy, are major obstacles to improve the quality of services in IIoT. In this paper, we propose the Digital Twin Edge Networks (DITEN) by incorporating digital twin into edge networks to fill the gap between physical systems and digital spaces. We further leverage the federated learning to construct digital twin models of IoT devices based on their running data. Moreover, to mitigate the communication overhead, we propose an asynchronous model update scheme and formulate the federated learning scheme as an optimization problem. We further decompose the problem and solve the subproblems based on the Deep Neural Network (DNN) model. Numerical results show that our proposed federated learning scheme for DITEN improves the communication efficiency and reduces the transmission energy cost.

[1]  R. N. Uma,et al.  Optimal Joint Scheduling and Cloud Offloading for Mobile Applications , 2019, IEEE Transactions on Cloud Computing.

[2]  Albert Y. Zomaya,et al.  Federated Learning over Wireless Networks: Optimization Model Design and Analysis , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[3]  Chau Yuen,et al.  A Novel Framework of Three-Hierarchical Offloading Optimization for MEC in Industrial IoT Networks , 2020, IEEE Transactions on Industrial Informatics.

[4]  Yan Zhang,et al.  Blockchain and Federated Learning for Privacy-Preserved Data Sharing in Industrial IoT , 2020, IEEE Transactions on Industrial Informatics.

[5]  Chau Yuen,et al.  Cluster Pruning: An Efficient Filter Pruning Method for Edge AI Vision Applications , 2020, IEEE Journal of Selected Topics in Signal Processing.

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[8]  Zheng Chang,et al.  Dynamic Resource Allocation and Computation Offloading for IoT Fog Computing System , 2020, IEEE Transactions on Industrial Informatics.

[9]  Canh Dinh,et al.  Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation , 2019, IEEE/ACM Transactions on Networking.

[10]  Dimitris S. Papailiopoulos,et al.  ATOMO: Communication-efficient Learning via Atomic Sparsification , 2018, NeurIPS.

[11]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[12]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[13]  Edward H. Glaessgen,et al.  The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles , 2012 .

[14]  Ke Zhang,et al.  Deep Learning Empowered Task Offloading for Mobile Edge Computing in Urban Informatics , 2019, IEEE Internet of Things Journal.

[15]  이창기,et al.  Convolutional Neural Network를 이용한 한국어 영화평 감성 분석 , 2016 .

[16]  Bo Yang,et al.  Computation Offloading in Multi-Access Edge Computing: A Multi-Task Learning Approach , 2021, IEEE Transactions on Mobile Computing.

[17]  Sebastian Caldas,et al.  Expanding the Reach of Federated Learning by Reducing Client Resource Requirements , 2018, ArXiv.

[18]  Mark W. Schmidt,et al.  Hybrid Deterministic-Stochastic Methods for Data Fitting , 2011, SIAM J. Sci. Comput..

[19]  Daniele Tarchi,et al.  Multi-Objective Computation Sharing in Energy and Delay Constrained Mobile Edge Computing Environments , 2021, IEEE Transactions on Mobile Computing.

[20]  Geyong Min,et al.  Federated Learning Based Proactive Content Caching in Edge Computing , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[21]  Ke Zhang,et al.  Artificial Intelligence Inspired Transmission Scheduling in Cognitive Vehicular Communications and Networks , 2019, IEEE Internet of Things Journal.

[22]  Mehdi Bennis,et al.  Multi-Tenant Cross-Slice Resource Orchestration: A Deep Reinforcement Learning Approach , 2018, IEEE Journal on Selected Areas in Communications.

[23]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[24]  Yang Song,et al.  Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning , 2018, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[25]  Ke Zhang,et al.  Edge Intelligence and Blockchain Empowered 5G Beyond for the Industrial Internet of Things , 2019, IEEE Network.

[26]  Song Han,et al.  Industrial Internet of Things: Challenges, Opportunities, and Directions , 2018, IEEE Transactions on Industrial Informatics.

[27]  Boi Faltings,et al.  Federated Learning with Bayesian Differential Privacy , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[28]  Tapani Ristaniemi,et al.  Incentive Mechanism for Edge-Computing-Based Blockchain , 2020, IEEE Transactions on Industrial Informatics.

[29]  Sha Wei,et al.  A Digital Twin-Based Approach for Quality Control and Optimization of Complex Product Assembly , 2019, 2019 International Conference on Artificial Intelligence and Advanced Manufacturing (AIAM).

[30]  Dongning Guo,et al.  Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks , 2018, IEEE Journal on Selected Areas in Communications.

[31]  He Zhang,et al.  Digital Twin in Industry: State-of-the-Art , 2019, IEEE Transactions on Industrial Informatics.

[32]  Takayuki Nishio,et al.  Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[33]  Ke Zhang,et al.  Deep Reinforcement Learning and Permissioned Blockchain for Content Caching in Vehicular Edge Computing and Networks , 2020, IEEE Transactions on Vehicular Technology.

[34]  Mohsen Guizani,et al.  Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services , 2018, IEEE Internet of Things Journal.

[35]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2021, Found. Trends Mach. Learn..