A framework for energy and carbon footprint analysis of distributed and federated edge learning

Recent advances in distributed learning raise environmental concerns due to the large energy needed to train and move data to/from data centers. Novel paradigms, such as federated learning (FL), are suitable for decentralized model training across devices or silos that simultaneously act as both data producers and learners. Unlike centralized learning (CL) techniques, relying on big-data fusion and analytics located in energy hungry data centers, in FL scenarios devices collaboratively train their models without sharing their private data. This article breaks down and analyzes the main factors that influence the environmental footprint of FL policies compared with classical CL/Big-Data algorithms running in data centers. The proposed analytical framework takes into account both learning and communication energy costs, as well as the incurred greenhouse gas, or carbon equivalent, emissions. The framework is evaluated in an industrial setting assuming a real-world robotized workplace. Results show that FL allows remarkable end-to-end energy savings (30%÷40%) in low-rate/power IoT communications (with limited energy efficiency). On the other hand, FL is slower to converge when local data are unevenly distributed (often 2x slower than CL).

[1]  J. Koomey,et al.  Characteristics of low-carbon data centres , 2013 .

[2]  Alfonso Capozzoli,et al.  Cooling Systems in Data Centers: State of Art and Emerging Technologies , 2015 .

[3]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[4]  Matthieu Cord,et al.  Gossip training for deep learning , 2016, ArXiv.

[5]  Yonggang Wen,et al.  Data Center Energy Consumption Modeling: A Survey , 2016, IEEE Communications Surveys & Tutorials.

[6]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[7]  Emil Björnson,et al.  How Energy-Efficient Can a Wireless Communication System Become? , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[8]  Monica Nicoli,et al.  Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks , 2019, IEEE Internet of Things Journal.

[9]  Thomas Watteyne,et al.  IETF 6TiSCH: A Tutorial , 2020, IEEE Communications Surveys & Tutorials.

[10]  Yonina C. Eldar,et al.  Federated Learning with Quantization Constraints , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Osvaldo Simeone,et al.  Decentralized Federated Learning via SGD over Wireless D2D Networks , 2020, 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[12]  Titouan Parcollet,et al.  Can Federated Learning Save The Planet , 2020 .

[13]  Mehdi Bennis,et al.  A Joint Decentralized Federated Learning and Communications Framework for Industrial Networks , 2020, 2020 IEEE 25th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD).

[14]  M. Bennis,et al.  GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning , 2019, J. Mach. Learn. Res..

[15]  Zhang Ningbo,et al.  Energy Efficiency for NPUSCH in NB-IoT with Guard Band , 2020 .

[16]  Deniz Gündüz,et al.  Federated Learning Over Wireless Fading Channels , 2019, IEEE Transactions on Wireless Communications.

[17]  Raghavendra Selvan,et al.  Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models , 2020, ArXiv.

[18]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[19]  S. Frank,et al.  A review of successful climate change mitigation policies in major emitting economies and the potential of global replication , 2021, Renewable and Sustainable Energy Reviews.

[20]  Mehdi Bennis,et al.  Opportunities of Federated Learning in Connected, Cooperative, and Automated Industrial Systems , 2021, IEEE Communications Magazine.

[21]  Zheng Chen,et al.  Consensus-Based Distributed Computation of Link-Based Network Metrics , 2020, ArXiv.

[22]  Manuel Beschi,et al.  A Multisensory Edge-Cloud Platform for Opportunistic Radio Sensing in Cobot Environments , 2021, IEEE Internet of Things Journal.