Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks

Implementing federated learning (FL) algorithms in wireless networks has garnered a wide range of attention. However, few works have considered the impact of user mobility on the learning performance. To fill this research gap, firstly, we develop a theoretical model to characterize the hierarchical federated learning (HFL) algorithm in wireless networks where the mobile users may roam across multiple edge access points (APs), leading to incompletion of inconsistent FL training. Secondly, we provide the convergence analysis of HFL with user mobility. Our analysis proves that the learning performance of HFL deteriorates drastically with highly-mobile users. And this decline in the learning performance will be exacerbated with small number of participants and large data distribution divergences among user’s local data. To circumvent these issues, we propose a mobility-aware cluster federated learning (MACFL) algorithm by redesigning the access mechanism, local update rule and model aggregation scheme. Finally, we provide experiments to evaluate the learning performance of HFL and our MACFL. The results show that our MACFL can enhance the learning performance, especially for three different cases: the case of users with non-independent and identical distribution (non-IID) data, the case of users with high mobility, and the cases with a small number of users.

[1]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[2]  Deniz Gündüz,et al.  Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Yijun Huang,et al.  Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization , 2015, NIPS.

[4]  H. Vincent Poor,et al.  Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge , 2021, IEEE Transactions on Wireless Communications.

[5]  Yonina C. Eldar,et al.  The Communication-Aware Clustered Federated Learning Problem , 2020, 2020 IEEE International Symposium on Information Theory (ISIT).

[6]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[7]  Wojciech Samek,et al.  Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Ali H. Sayed,et al.  Variance-Reduced Stochastic Learning by Networked Agents Under Random Reshuffling , 2017, IEEE Transactions on Signal Processing.

[9]  Christopher Briggs,et al.  Federated learning with hierarchical clustering of local updates to improve training on non-IID data , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[10]  Jun Zhang,et al.  Edge-Assisted Hierarchical Federated Learning with Non-IID Data , 2019, ArXiv.

[11]  Dusit Niyato,et al.  Resource Allocation in Mobility-Aware Federated Learning Networks: A Deep Reinforcement Learning Approach , 2019, 2020 IEEE 6th World Forum on Internet of Things (WF-IoT).

[12]  Nguyen H. Tran,et al.  Personalized Federated Learning with Moreau Envelopes , 2020, NeurIPS.

[13]  Rong-Rong Chen,et al.  Local Averaging Helps: Hierarchical Federated Learning and Convergence Analysis , 2020, ArXiv.

[14]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[15]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[16]  Qiong Wu,et al.  HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning , 2020, IEEE Transactions on Wireless Communications.

[17]  H. Vincent Poor,et al.  Scheduling Policies for Federated Learning in Wireless Networks , 2019, IEEE Transactions on Communications.

[18]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[19]  Xu Chen,et al.  In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning , 2018, IEEE Network.

[20]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[21]  Stacy Patterson,et al.  Multi-Level Local SGD for Heterogeneous Hierarchical Networks , 2020, ArXiv.

[22]  Jianyu Wang,et al.  Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms , 2018, ArXiv.

[23]  M. Shamim Hossain,et al.  Mobility-Aware Proactive Edge Caching for Connected Vehicles Using Federated Learning , 2021, IEEE Transactions on Intelligent Transportation Systems.

[24]  Cheng Chen,et al.  FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling , 2020, 2020 IEEE International Conference on Big Data (Big Data).

[25]  Jian Pei,et al.  Personalized Cross-Silo Federated Learning on Non-IID Data , 2020, AAAI.

[26]  Aryan Mokhtari,et al.  Personalized Federated Learning: A Meta-Learning Approach , 2020, ArXiv.

[27]  Walid Saad,et al.  A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks , 2021, IEEE Transactions on Wireless Communications.

[28]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[29]  V. Bezborodov.  Markov birth-and-death dynamics of populations , 2015, Мiждисциплiнарнi дослiдження складних систем.

[30]  H. Vincent Poor,et al.  Convergence Time Optimization for Federated Learning Over Wireless Networks , 2020, IEEE Transactions on Wireless Communications.

[31]  Jean-Philippe Vert,et al.  Clustered Multi-Task Learning: A Convex Formulation , 2008, NIPS.