Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks

With growth in the number of smart devices and advancements in their hardware, in recent years, data-driven machine learning techniques have drawn significant attention. However, due to privacy and communication issues, it is not possible to collect this data at a centralized location. Federated learning is a machine learning setting where the centralized location trains a learning model over remote devices. Federated learning algorithms cannot be employed in the real world scenarios unless they consider unreliable and resource-constrained nature of the wireless medium. In this paper, we propose a federated learning algorithm that is suitable for cellular wireless networks. We prove its convergence, and provide the optimal scheduling policy that maximizes the convergence rate. We also study the effect of local computation steps and communication steps on the convergence of the proposed algorithm. We prove, in practice, federated learning algorithms may solve a different problem than the one that they have been employed for if the unreliability of wireless channels is neglected. Finally, through numerous experiments on real and synthetic datasets, we demonstrate the convergence of our proposed algorithm.

[1]  Fan Zhou,et al.  On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimization , 2017, IJCAI.

[2]  Marco Di Renzo,et al.  Stochastic Geometry Modeling and System-Level Analysis of Uplink Heterogeneous Cellular Networks With Multi-Antenna Base Stations , 2016, IEEE Transactions on Communications.

[3]  N. Kazarinoff Analytic Inequalities , 2021, Inequalities in Analysis and Probability.

[4]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[5]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2021, Found. Trends Mach. Learn..

[6]  Robert W. Heath,et al.  Analyzing Uplink SINR and Rate in Massive MIMO Systems Using Stochastic Geometry , 2015, IEEE Transactions on Communications.

[7]  Jeffrey G. Andrews,et al.  Tractable Model for Rate in Self-Backhauled Millimeter Wave Cellular Networks , 2014, IEEE Journal on Selected Areas in Communications.

[8]  Sebastian U. Stich,et al.  Local SGD Converges Fast and Communicates Little , 2018, ICLR.

[9]  Xiang Li,et al.  On the Convergence of FedAvg on Non-IID Data , 2019, ICLR.

[10]  Martin Haenggi,et al.  Stochastic Geometry for Wireless Networks , 2012 .

[11]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[12]  Sashank J. Reddi,et al.  SCAFFOLD: Stochastic Controlled Averaging for Federated Learning , 2019, ICML.

[13]  Ekram Hossain,et al.  On Stochastic Geometry Modeling of Cellular Uplink Transmission With Truncated Channel Inversion Power Control , 2014, IEEE Transactions on Wireless Communications.

[14]  Martin Haenggi,et al.  The Meta Distribution of the SIR for Cellular Networks With Power Control , 2017, IEEE Transactions on Communications.

[15]  Farzin Haddadpour,et al.  On the Convergence of Local Descent Methods in Federated Learning , 2019, ArXiv.

[16]  Michael I. Jordan,et al.  CoCoA: A General Framework for Communication-Efficient Distributed Optimization , 2016, J. Mach. Learn. Res..

[17]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[18]  Martin J. Wainwright,et al.  Communication-efficient algorithms for statistical optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[19]  Mung Chiang,et al.  Fast-Convergent Federated Learning , 2020, IEEE Journal on Selected Areas in Communications.

[20]  Walid Saad,et al.  Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges , 2020, IEEE Communications Surveys & Tutorials.

[21]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[22]  H. Vincent Poor,et al.  Scheduling Policies for Federated Learning in Wireless Networks , 2019, IEEE Transactions on Communications.

[23]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[24]  Deniz Gündüz,et al.  Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[25]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[26]  Canh Dinh,et al.  Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation , 2019, IEEE/ACM Transactions on Networking.

[27]  Ohad Shamir,et al.  Communication-Efficient Distributed Optimization using an Approximate Newton-type Method , 2013, ICML.

[28]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.