Scheduling in Cellular Federated Edge Learning with Importance and Channel Awareness

In cellular federated edge learning (FEEL), multiple edge devices holding local data jointly train a neural network by communicating learning updates with an access point without exchanging their data samples. With very limited communication resources, it is beneficial to schedule the most informative local learning updates. This paper focuses on FEEL with gradient averaging over participating devices in each round of communication. A novel scheduling policy is proposed to exploit both diversity in multiuser channels and diversity in the “importance” of the edge devices’ learning updates. First, a new probabilistic scheduling framework is developed to yield unbiased update aggregation in FEEL. The importance of a local learning update is measured by its gradient divergence. If one edge device is scheduled in each communication round, the scheduling policy is derived in closed form to achieve the optimal trade-off between channel quality and update importance. The probabilistic scheduling framework is then extended to allow scheduling multiple edge devices in each communication round. Numerical results obtained using popular models and learning datasets demonstrate that the proposed scheduling policy can achieve faster model convergence and higher learning accuracy than conventional scheduling policies that only exploit a single type of diversity.

[1]  Howard H. Yang,et al.  Federated-Learning-Enabled Intelligent Fog Radio Access Networks: Fundamental Theory, Key Techniques, and Future Trends , 2020, IEEE Wireless Communications.

[2]  Mehdi Bennis,et al.  Federated Learning under Channel Uncertainty: Joint Client Scheduling and Resource Allocation , 2020, 2020 IEEE Wireless Communications and Networking Conference (WCNC).

[3]  S. Kulkarni,et al.  Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge , 2020, IEEE Transactions on Wireless Communications.

[4]  H. Vincent Poor,et al.  Update Aware Device Scheduling for Federated Learning at the Wireless Edge , 2020, 2020 IEEE International Symposium on Information Theory (ISIT).

[5]  H. Vincent Poor,et al.  Convergence Time Optimization for Federated Learning Over Wireless Networks , 2020, IEEE Transactions on Wireless Communications.

[6]  Deniz Gündüz,et al.  One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis , 2020, IEEE Transactions on Wireless Communications.

[7]  H. Vincent Poor,et al.  Performance Optimization of Federated Learning over Wireless Networks , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[8]  Kaibin Huang,et al.  An Overview of Data-Importance Aware Radio Resource Management for Edge Machine Learning , 2019, Journal of Communications and Information Networks.

[9]  Choong Seon Hong,et al.  Energy Efficient Federated Learning Over Wireless Communication Networks , 2019, IEEE Transactions on Wireless Communications.

[10]  H. Vincent Poor,et al.  Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Jun Zhang,et al.  Data-Importance Aware User Scheduling for Communication-Efficient Edge Machine Learning , 2019, IEEE Transactions on Cognitive Communications and Networking.

[12]  Deniz Gündüz,et al.  Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  H. Vincent Poor,et al.  Scheduling Policies for Federated Learning in Wireless Networks , 2019, IEEE Transactions on Communications.

[14]  Xiang Li,et al.  On the Convergence of FedAvg on Non-IID Data , 2019, ICLR.

[15]  Kaibin Huang,et al.  Wireless Data Acquisition for Edge Learning: Importance-Aware Retransmission , 2019, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[16]  Guanding Yu,et al.  Accelerating DNN Training in Wireless Federated Edge Learning Systems , 2019, IEEE Journal on Selected Areas in Communications.

[17]  Albert Y. Zomaya,et al.  Federated Learning over Wireless Networks: Optimization Model Design and Analysis , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[18]  Walid Saad,et al.  A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems , 2019, IEEE Network.

[19]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[20]  Zhi Ding,et al.  Federated Learning via Over-the-Air Computation , 2018, IEEE Transactions on Wireless Communications.

[21]  Kaibin Huang,et al.  Broadband Analog Aggregation for Low-Latency Federated Edge Learning , 2018, IEEE Transactions on Wireless Communications.

[22]  Mehdi Bennis,et al.  Wireless Network Intelligence at the Edge , 2018, Proceedings of the IEEE.

[23]  Xu Chen,et al.  In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning , 2018, IEEE Network.

[24]  Kaibin Huang,et al.  Towards an Intelligent Edge: Wireless Communication Meets Machine Learning , 2018, ArXiv.

[25]  Qi Hao,et al.  Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey , 2018, IEEE Communications Surveys & Tutorials.

[26]  Georgios B. Giannakis,et al.  LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning , 2018, NeurIPS.

[27]  Klaus-Robert Müller,et al.  Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[28]  Takayuki Nishio,et al.  Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[29]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[30]  Hamed Haddadi,et al.  Deep Learning in Mobile and Wireless Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[31]  William J. Dally,et al.  Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training , 2017, ICLR.

[32]  Ursula Challita,et al.  Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial , 2017, IEEE Communications Surveys & Tutorials.

[33]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[34]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[35]  Tong Zhang,et al.  Stochastic Optimization with Importance Sampling for Regularized Loss Minimization , 2014, ICML.

[36]  Mark W. Schmidt,et al.  Hybrid Deterministic-Stochastic Methods for Data Fitting , 2011, SIAM J. Sci. Comput..

[37]  Jeffrey G. Andrews,et al.  Adaptive resource allocation in multiuser OFDM systems with proportional rate constraints , 2005, IEEE Transactions on Wireless Communications.

[38]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[39]  Pravin Varaiya,et al.  QoS aware adaptive resource allocation techniques for fair scheduling in OFDMA based broadband wireless access systems , 2003, IEEE Trans. Broadcast..