Energy Efficient Federated Learning Over Wireless Communication Networks

In this paper, the problem of energy efficient transmission and computation resource allocation for federated learning (FL) over wireless communication networks is investigated. In the considered model, each user exploits limited local computational resources to train a local FL model with its collected data and, then, sends the trained FL model parameters to a base station (BS) which aggregates the local FL model and broadcasts it back to all of the users. Since FL involves an exchange of a learning model between users and the BS, both computation and communication latencies are determined by the learning accuracy level. Meanwhile, due to the limited energy budget of the wireless users, both local computation energy and transmission energy must be considered during the FL process. This joint learning and communication problem is formulated as an optimization problem whose goal is to minimize a weighted sum of the completion time of FL, local computation energy, and transmission energy of all users, that captures the tradeoff of latency and energy consumption for FL. To solve this problem, an iterative algorithm is proposed where, at every step, closed-form solutions for time allocation, bandwidth allocation, power control, computation frequency, and learning accuracy are derived. For the special case that only minimizes the completion time, a bisection-based algorithm is proposed to obtain the optimal solution. Numerical results show that the proposed algorithms can reduce up to 25.6% delay and 37.6% energy consumption compared to conventional FL methods.

[1]  Hamidou Tembine Distributed Strategic Learning for Wireless Engineers , 2017 .

[2]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[3]  Walid Saad,et al.  A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems , 2019, IEEE Network.

[4]  Walid Saad,et al.  Federated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks , 2018, IEEE Transactions on Wireless Communications.

[5]  Kin K. Leung,et al.  When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[6]  Albert Y. Zomaya,et al.  Federated Learning over Wireless Networks: Optimization Model Design and Analysis , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[7]  Mihaela van der Schaar,et al.  Machine Learning in the Air , 2019, IEEE Journal on Selected Areas in Communications.

[8]  H. Vincent Poor,et al.  Scheduling Policies for Federated Learning in Wireless Networks , 2019, IEEE Transactions on Communications.

[9]  Ohad Shamir,et al.  Communication-Efficient Distributed Optimization using an Approximate Newton-type Method , 2013, ICML.

[10]  Shuguang Cui,et al.  Delay Minimization for Federated Learning Over Wireless Communication Networks , 2020, ArXiv.

[11]  Yong Wang,et al.  Low-Latency Broadband Analog Aggregation for Federated Edge Learning , 2018, ArXiv.

[12]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[13]  Werner Dinkelbach On Nonlinear Fractional Programming , 1967 .

[14]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[15]  Arumugam Nallanathan,et al.  Joint Blocklength and Location Optimization for URLLC-Enabled UAV Relay Systems , 2019, IEEE Communications Letters.

[16]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.

[17]  Walid Saad,et al.  Mobile Unmanned Aerial Vehicles (UAVs) for Energy-Efficient Internet of Things Communications , 2017, IEEE Transactions on Wireless Communications.

[18]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[19]  Seong-Lyun Kim,et al.  Multi-hop Federated Private Data Augmentation with Sample Compression , 2019, ArXiv.

[20]  Hien Quoc Ngo,et al.  Cell-Free Massive MIMO for Wireless Federated Learning , 2019, IEEE Transactions on Wireless Communications.

[21]  Walid Saad,et al.  Caching in the Sky: Proactive Deployment of Cache-Enabled Unmanned Aerial Vehicles for Optimized Quality-of-Experience , 2016, IEEE Journal on Selected Areas in Communications.

[22]  Ursula Challita,et al.  Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial , 2017, IEEE Communications Surveys & Tutorials.

[23]  Deniz Gündüz,et al.  Energy-Aware Analog Aggregation for Federated Learning with Redundant Data , 2019, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[24]  Zhi Ding,et al.  Federated Learning via Over-the-Air Computation , 2018, IEEE Transactions on Wireless Communications.

[25]  Mehdi Bennis,et al.  Wireless Network Intelligence at the Edge , 2018, Proceedings of the IEEE.

[26]  Kin K. Leung,et al.  Energy-Efficient Radio Resource Allocation for Federated Edge Learning , 2019, 2020 IEEE International Conference on Communications Workshops (ICC Workshops).

[27]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[28]  Walid Saad,et al.  A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks , 2021, IEEE Transactions on Wireless Communications.

[29]  Kaibin Huang,et al.  Towards an Intelligent Edge: Wireless Communication Meets Machine Learning , 2018, ArXiv.

[30]  Khaled Ben Letaief,et al.  Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices , 2016, IEEE Journal on Selected Areas in Communications.

[31]  Joonhyuk Kang,et al.  Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data , 2019, 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[32]  Walid Saad,et al.  Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications , 2018, IEEE Transactions on Communications.