Green Federated Learning Over Cloud-RAN with Limited Fronthual Capacity and Quantized Neural Networks

In this paper, we propose an energy-efficient federated learning (FL) framework for the energy-constrained devices over cloud radio access network (Cloud-RAN), where each device adopts quantized neural networks (QNNs) to train a local FL model and transmits the quantized model parameter to the remote radio heads (RRHs). Each RRH receives the signals from devices over the wireless link and forwards the signals to the server via the fronthaul link. We rigorously develop an energy consumption model for the local training at devices through the use of QNNs and communication models over Cloud-RAN. Based on the proposed energy consumption model, we formulate an energy minimization problem that optimizes the fronthaul rate allocation, user transmit power allocation, and QNN precision levels while satisfying the limited fronthaul capacity constraint and ensuring the convergence of the proposed FL model to a target accuracy. To solve this problem, we analyze the convergence rate and propose efficient algorithms based on the alternative optimization technique. Simulation results show that the proposed FL framework can significantly reduce energy consumption compared to other conventional approaches. We draw the conclusion that the proposed framework holds great potential for achieving a sustainable and environmentally-friendly FL in Cloud-RAN.

[1]  Yuanming Shi,et al.  Task-Oriented Communications for 6G: Vision, Principles, and Technologies , 2023, IEEE Wireless Communications.

[2]  Zhiqiang Hao,et al.  A game-theoretic approach for federated learning: A trade-off among privacy, accuracy and energy , 2023, Digital Communications and Networks.

[3]  S. Chatzinotas,et al.  Sparsification and Optimization for Energy-Efficient Federated Learning in Wireless Edge Networks , 2022, GLOBECOM 2022 - 2022 IEEE Global Communications Conference.

[4]  Hejiao Huang,et al.  Resource Optimization and Device Scheduling for Flexible Federated Edge Learning with Tradeoff Between Energy Consumption and Model Performance , 2022, Mobile Networks and Applications.

[5]  M. Debbah,et al.  Green, Quantized Federated Learning Over Wireless Networks: An Energy-Efficient Design , 2022, IEEE Transactions on Wireless Communications.

[6]  M. Bennis,et al.  An Energy and Carbon Footprint Analysis of Distributed and Federated Learning , 2022, IEEE Transactions on Green Communications and Networking.

[7]  Yuanming Shi,et al.  Reconfigurable Intelligent Surfaces Empowered Green Wireless Networks With User Admission Control , 2022, IEEE Transactions on Communications.

[8]  Colin N. Jones,et al.  Over-the-Air Federated Learning via Second-Order Optimization , 2022, IEEE Transactions on Wireless Communications.

[9]  S. Buzzi,et al.  The Promising Marriage of Mobile Edge Computing and Cell-Free Massive MIMO , 2022, ICC 2022 - IEEE International Conference on Communications.

[10]  Minh N. Dao,et al.  Energy-Efficient Massive MIMO for Federated Learning: Transmission Designs and Resource Allocations , 2021, IEEE Open Journal of the Communications Society.

[11]  K. B. Letaief,et al.  Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications , 2021, IEEE Journal on Selected Areas in Communications.

[12]  M. Debbah,et al.  On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks , 2021, ICC 2022 - IEEE International Conference on Communications.

[13]  Qingjiang Shi,et al.  Quantized Federated Learning Under Transmission Delay and Outage Constraints , 2021, IEEE Journal on Selected Areas in Communications.

[14]  Yan Huo,et al.  1-Bit Compressive Sensing for Efficient Federated Learning Over the Air , 2021, IEEE Transactions on Wireless Communications.

[15]  Xiaojun Yuan,et al.  Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning , 2021, IEEE Journal on Selected Areas in Communications.

[16]  Leandros Tassiulas,et al.  Cost-Effective Federated Learning Design , 2020, IEEE INFOCOM 2021 - IEEE Conference on Computer Communications.

[17]  Cong Shen,et al.  Design and Analysis of Uplink and Downlink Communications for Federated Learning , 2020, IEEE Journal on Selected Areas in Communications.

[18]  Zhibin Wang,et al.  Federated Learning via Intelligent Reflecting Surface , 2020, IEEE Transactions on Wireless Communications.

[19]  Deniz Gündüz,et al.  Federated Learning With Quantized Global Model Updates , 2020, ArXiv.

[20]  H. Dai,et al.  Communication Efficient Federated Learning with Energy Awareness over Wireless Networks , 2020, IEEE Transactions on Wireless Communications.

[21]  Jun Li,et al.  A Compressive Sensing Approach for Federated Learning Over Massive MIMO Communication Systems , 2020, IEEE Transactions on Wireless Communications.

[22]  Jun Zhang,et al.  Communication-Efficient Edge AI: Algorithms and Systems , 2020, IEEE Communications Surveys & Tutorials.

[23]  Mohammad Javad Emadi,et al.  Performance Analysis of Cell-Free Massive MIMO System With Limited Fronthaul Capacity and Hardware Impairments , 2020, IEEE Transactions on Wireless Communications.

[24]  Choong Seon Hong,et al.  Energy Efficient Federated Learning Over Wireless Communication Networks , 2019, IEEE Transactions on Wireless Communications.

[25]  Solmaz Niknam,et al.  Federated Learning for Wireless Communications: Motivation, Opportunities, and Challenges , 2019, IEEE Communications Magazine.

[26]  Deniz Gündüz,et al.  Federated Learning Over Wireless Fading Channels , 2019, IEEE Transactions on Wireless Communications.

[27]  Xiang Li,et al.  On the Convergence of FedAvg on Non-IID Data , 2019, ICLR.

[28]  Deniz Gündüz,et al.  Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[29]  Anastasios Kyrillidis,et al.  Compressing Gradient Optimizers via Count-Sketches , 2019, ICML.

[30]  Qiang Yang,et al.  Federated Machine Learning , 2019, ACM Trans. Intell. Syst. Technol..

[31]  Zhi Ding,et al.  Federated Learning via Over-the-Air Computation , 2018, IEEE Transactions on Wireless Communications.

[32]  Qiang Li,et al.  Min-Max Latency Optimization for Multiuser Computation Offloading in Fog-Radio Access Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Wei Yu,et al.  Fractional Programming for Communication Systems—Part I: Power Control and Beamforming , 2018, IEEE Transactions on Signal Processing.

[34]  Marian Verhelst,et al.  Minimum energy quantized neural networks , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[35]  Erik G. Larsson,et al.  On the Total Energy Efficiency of Cell-Free Massive MIMO , 2017, IEEE Transactions on Green Communications and Networking.

[36]  Marian Verhelst,et al.  14.5 Envision: A 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable Convolutional Neural Network processor in 28nm FDSOI , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[37]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..

[38]  Rui Zhang,et al.  Joint Millimeter-Wave Fronthaul and OFDMA Resource Allocation in Ultra-Dense CRAN , 2016, IEEE Transactions on Communications.

[39]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[40]  Jakub Konecný,et al.  Federated Optimization: Distributed Optimization Beyond the Datacenter , 2015, ArXiv.

[41]  Liang Liu,et al.  Joint Power Control and Fronthaul Rate Allocation for Throughput Maximization in OFDMA-Based Cloud Radio Access Network , 2014, IEEE Transactions on Communications.

[42]  Yuanming Shi,et al.  Group Sparse Beamforming for Green Cloud-RAN , 2013, IEEE Transactions on Wireless Communications.

[43]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[44]  Moses Charikar,et al.  Finding frequent items in data streams , 2002, Theor. Comput. Sci..

[45]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[46]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[47]  Vladimir Braverman,et al.  FetchSGD: Communication-Efficient Federated Learning with Sketching , 2022 .

[48]  N. Lee,et al.  Communication-Efficient Federated Learning via Quantized Compressed Sensing , 2021, IEEE Transactions on Wireless Communications.