A Joint Decentralized Federated Learning and Communications Framework for Industrial Networks

Industrial wireless networks are pushing towards distributed architectures moving beyond traditional server-client transactions. Paired with this trend, new synergies are emerging among sensing, communications and Machine Learning (ML) co-design, where resources need to be distributed across different wireless field devices, acting as both data producers and learners. Considering this landscape, Federated Learning (FL) solutions are suitable for training a ML model in distributed systems. In particular, decentralized FL policies target scenarios where learning operations must be implemented collaboratively, without relying on the server, and by exchanging model parameters updates rather than training data over capacity-constrained radio links. This paper proposes a real-time framework for the analysis of decentralized FL systems running on top of industrial wireless networks rooted in the popular Time Slotted Channel Hopping (TSCH) radio interface of the IEEE 802.15.4e standard. The proposed framework is suitable for neural networks trained via distributed Stochastic Gradient Descent (SGD), it quantifies the effects of model pruning, sparsification and quantization, as well as physical and link layer constraints, on FL convergence time and learning loss. The goal is to set the fundamentals for comprehensive methods and procedures supporting decentralized FL pre-deployment design. The proposed tool can be thus used to optimize the deployment of the wireless network and the ML model before its actual installation. It has been verified based on real data targeting smart robotic-assisted manufacturing.

[1]  Monica Nicoli,et al.  Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks , 2019, IEEE Internet of Things Journal.

[2]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[3]  Deniz Gündüz,et al.  Federated Learning Over Wireless Fading Channels , 2019, IEEE Transactions on Wireless Communications.

[4]  Amitabha Ghosh,et al.  5G Evolution: A View on 5G Cellular Technology Beyond 3GPP Release 15 , 2019, IEEE Access.

[5]  Abhinav Vishnu,et al.  GossipGraD: Scalable Deep Learning using Gossip Communication based Asynchronous Gradient Descent , 2018, ArXiv.

[6]  Emiliano Sisinni,et al.  A Wireless Cloud Network Platform for Industrial Process Automation: Critical Data Publishing and Distributed Sensing , 2017, IEEE Transactions on Instrumentation and Measurement.

[7]  Thomas Watteyne,et al.  IETF 6TiSCH: A Tutorial , 2020, IEEE Communications Surveys & Tutorials.

[8]  Osvaldo Simeone,et al.  Decentralized Federated Learning via SGD over Wireless D2D Networks , 2020, 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[9]  H. Vincent Poor,et al.  Performance Optimization of Federated Learning over Wireless Networks , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[10]  Dan Alistarh,et al.  QSGD: Communication-Optimal Stochastic Gradient Descent, with Applications to Training Neural Networks , 2016, 1610.02132.

[11]  Diego Dujovne,et al.  6TiSCH Minimal Scheduling Function (MSF) , 2020, RFC.

[12]  H. Vincent Poor,et al.  Ultrareliable and Low-Latency Wireless Communication: Tail, Risk, and Scale , 2018, Proceedings of the IEEE.

[13]  Mehdi Bennis,et al.  GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning , 2019, J. Mach. Learn. Res..

[14]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[15]  Yonina C. Eldar,et al.  Federated Learning with Quantization Constraints , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Eduardo Tovar,et al.  IEEE 802.15.4e in a Nutshell: Survey and Performance Evaluation , 2018, IEEE Communications Surveys & Tutorials.

[17]  Thomas Watteyne,et al.  Orchestra: Robust Mesh Networks Through Autonomously Scheduled TSCH , 2015, SenSys.

[18]  Matthieu Cord,et al.  Gossip training for deep learning , 2016, ArXiv.

[19]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[20]  N. Abreu.  Old and new results on algebraic connectivity of graphs , 2007 .

[21]  Umberto Spagnolini,et al.  Consensus-Based Algorithms for Distributed Network-State Estimation and Localization , 2017, IEEE Transactions on Signal and Information Processing over Networks.