Socially-Aware-Clustering-Enabled Federated Learning for Edge Networks

Edge Intelligence based on federated learning (FL) can be considered to be a promising paradigm for many emerging, strict latency Internet of Things (IoT) applications. Furthermore, a rapid upsurge in the number of IoT devices is expected in the foreseeable future. Although FL enables privacy-preserving, on-device machine learning, it still exhibits a privacy leakage issue. A malicious aggregation server can infer the sensitive information of other end-devices using their local learning model updates. Furthermore, centralized FL aggregation server might stop working due to security attack or a physical damage. To address the aforementioned issues, we propose a novel concept of socially-aware-clustering-enabled dispersed FL. First, we present a novel framework for socially-aware-clustering-enabled dispersed FL. Second, we formulate a problem for minimizing the loss function of the proposed FL scheme. Third, we decompose the formulated problem into three sub-problems, such as local devices relative accuracy minimization (i.e., end-devices local accuracy maximization) sub-problem, clustering sub-problem, and resource allocation sub-problem, due to the NP-hard nature of the formulated problem. The clustering and resource allocation sub-problems are solved using low complexity schemes based on a matching theory. The end devices’ relative accuracy minimization problem is solved by using a convex optimizer. Finally, numerical results are provided for validation of the proposed FL scheme. Furthermore, we show the convergence of the proposed FL scheme for image classification tasks using the MNIST dataset.

[1]  Shivaram Narayanan,et al.  The Betweenness Centrality Of Biological Networks , 2005 .

[2]  Deniz Gündüz,et al.  Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  H. Vincent Poor,et al.  Wireless Communications for Collaborative Federated Learning , 2020, IEEE Communications Magazine.

[4]  Jun Zhang,et al.  Edge-Assisted Hierarchical Federated Learning with Non-IID Data , 2019, ArXiv.

[5]  Zhu Han,et al.  Matching Theory: Applications in wireless communications , 2016, IEEE Signal Processing Magazine.

[6]  Linyuan Lü,et al.  Predicting missing links via local information , 2009, 0901.0553.

[7]  Dan Liu,et al.  A Survey on Secure Data Analytics in Edge Computing , 2019, IEEE Internet of Things Journal.

[8]  Mads Haahr,et al.  Social network analysis for routing in disconnected delay-tolerant MANETs , 2007, MobiHoc '07.

[9]  Choong Seon Hong,et al.  A Crowdsourcing Framework for On-Device Federated Learning , 2020, IEEE Transactions on Wireless Communications.

[10]  Alister G. Burr,et al.  A General Upper Bound to Evaluate Packet Error Rate over Quasi-Static Fading Channels , 2011, IEEE Transactions on Wireless Communications.

[11]  Walid Saad,et al.  Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges , 2020, IEEE Communications Surveys & Tutorials.

[12]  Sheng Chen,et al.  Social-Aware Resource Allocation for Device-to-Device Communications Underlaying Cellular Networks , 2015, IEEE Wireless Communications Letters.

[13]  Albert Y. Zomaya,et al.  Federated Learning over Wireless Networks: Optimization Model Design and Analysis , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[14]  Nei Kato,et al.  Ten Challenges in Advancing Machine Learning Technologies toward 6G , 2020, IEEE Wireless Communications.

[15]  U. Brandes A faster algorithm for betweenness centrality , 2001 .

[16]  Zhu Han,et al.  6G Wireless Systems: A Vision, Architectural Elements, and Future Directions , 2020, IEEE Access.

[17]  Walid Saad,et al.  A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks , 2021, IEEE Transactions on Wireless Communications.

[18]  Amir Houmansadr,et al.  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[19]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[20]  Rongxing Lu,et al.  From Cloud to Fog Computing: A Review and a Conceptual Live VM Migration Framework , 2017, IEEE Access.

[21]  Ekram Hossain,et al.  Distributed Resource Allocation in 5G Cellular Networks , 2014, ArXiv.

[22]  Walid Saad,et al.  A Game-Theoretic Approach for Fair Coexistence Between LTE-U and Wi-Fi Systems , 2019, IEEE Transactions on Vehicular Technology.

[23]  Choong Seon Hong,et al.  Hierarchical Matching Game for Service Selection and Resource Purchasing in Wireless Network Virtualization , 2018, IEEE Communications Letters.

[24]  Sheng Chen,et al.  Social-aware D2D communications: qualitative insights and quantitative analysis , 2014, IEEE Communications Magazine.

[25]  Ying-Chang Liang,et al.  Federated Learning in Mobile Edge Networks: A Comprehensive Survey , 2020, IEEE Communications Surveys & Tutorials.

[26]  Walid Saad,et al.  Dynamic Clustering and User Association in Wireless Small-Cell Networks With Social Considerations , 2016, IEEE Transactions on Vehicular Technology.

[27]  Hanan Lutfiyya,et al.  Power-Aware Wireless Virtualized Resource Allocation with D2D Communication Underlaying LTE Network , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[28]  Walid Saad,et al.  Dispersed Federated Learning: Vision, Taxonomy, and Future Directions , 2020, IEEE Wireless Communications.

[29]  Lin Zhou On a conjecture by gale about one-sided matching problems , 1990 .

[30]  Atila Abdulkadiroglu,et al.  RANDOM SERIAL DICTATORSHIP AND THE CORE FROM RANDOM ENDOWMENTS IN HOUSE ALLOCATION PROBLEMS , 1998 .

[31]  Rong-Rong Chen,et al.  Local Averaging Helps: Hierarchical Federated Learning and Convergence Analysis , 2020, ArXiv.

[32]  Walid Saad,et al.  A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems , 2019, IEEE Network.

[33]  Mark W. Schmidt,et al.  Hybrid Deterministic-Stochastic Methods for Data Fitting , 2011, SIAM J. Sci. Comput..

[34]  Pasquale De Meo,et al.  A Novel Measure of Edge Centrality in Social Networks , 2012, Knowl. Based Syst..

[35]  Walid Saad,et al.  Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism , 2019, IEEE Communications Magazine.

[36]  Walid Saad,et al.  Mode Selection and Resource Allocation in Device-to-Device Communications: A Matching Game Approach , 2017, IEEE Transactions on Mobile Computing.

[37]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[38]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[39]  Zhu Han,et al.  Self Organizing Federated Learning Over Wireless Networks: A Socially Aware Clustering Approach , 2020, 2020 International Conference on Information Networking (ICOIN).

[40]  Zhu Han,et al.  A Survey on Socially Aware Device-to-Device Communications , 2018, IEEE Communications Surveys & Tutorials.