Can Federated Learning Save The Planet

Despite impressive results, deep learning-based technologies also raise severe privacy and environmental concerns induced by the training procedure often conducted in data centers. In response, alternatives to centralized training such as Federated Learning (FL) have emerged. Perhaps unexpectedly, FL in particular is starting to be deployed at a global scale by companies that must adhere to new legal demands and policies originating from governments and the civil society for privacy protection. However, the potential environmental impact related to FL remains unclear and unexplored. This paper offers the first-ever systematic study of the carbon footprint of FL. First, we propose a rigorous model to quantify the carbon footprint, hence facilitating the investigation of the relationship between FL design and carbon emissions. Then, we compare the carbon footprint of FL to traditional centralized learning. Finally, we highlight and connect the reported results to the future challenges and trends in FL to reduce its environmental impact, including algorithms efficiency, hardware capabilities, and stronger industry transparency.

[1]  Miro Hodak,et al.  Towards Power Efficiency in Deep Learning on Data Center Hardware , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[2]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[3]  Keqiu Li,et al.  Energy Consumption in Cloud Computing Data Centers , 2014, CloudCom 2014.

[4]  Crowley,et al.  Atmospheric science: Methane rises from wetlands , 2011, Nature.

[5]  J. Minx,et al.  Climate Change 2014 : Synthesis Report , 2014 .

[6]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[7]  Amar Phanishayee,et al.  The Non-IID Data Quagmire of Decentralized Machine Learning , 2019, ICML.

[8]  Keywan Riahi,et al.  Synthesis report: summary for policy makers , 2014 .

[9]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.

[10]  Takayuki Nishio,et al.  Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[11]  Xiang Li,et al.  On the Convergence of FedAvg on Non-IID Data , 2019, ICLR.

[12]  Peter Henderson,et al.  Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning , 2020, ArXiv.

[13]  Alexandre Lacoste,et al.  Quantifying the Carbon Emissions of Machine Learning , 2019, ArXiv.

[14]  Lars Kai Hansen,et al.  Towards Federated Learning: Robustness Analytics to Data Heterogeneity , 2020, ArXiv.

[15]  Alfonso Capozzoli,et al.  Cooling Systems in Data Centers: State of Art and Emerging Technologies , 2015 .

[16]  Andrew McCallum,et al.  Energy and Policy Considerations for Deep Learning in NLP , 2019, ACL.

[17]  Flávio Miguel Varejão,et al.  Monthly energy consumption forecast: A deep learning approach , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[18]  Raghavendra Selvan,et al.  Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models , 2020, ArXiv.