Energy-Efficient Multi-Orchestrator Mobile Edge Learning

Mobile Edge Learning (MEL) is a collaborative learning paradigm that features distributed training of Machine Learning (ML) models over edge devices (e.g., IoT devices). In MEL, possible coexistence of multiple learning tasks with different datasets may arise. The heterogeneity in edge devices’ capabilities will require the joint optimization of the learnersorchestrator association and task allocation. To this end, we aim to develop an energy-efficient framework for learners-orchestrator association and learning task allocation, in which each orchestrator gets associated with a group of learners with the same learning task based on their communication channel qualities and computational resources, and allocate the tasks accordingly. Therein, a multi-objective optimization problem is formulated to minimize the total energy consumption and maximize the learning tasks’ accuracy. However, solving such optimization problem requires centralization and the presence of the whole environment information at a single entity, which becomes impractical in large-scale systems. To reduce the solution complexity and to enable solution decentralization, we propose lightweight heuristic algorithms that can achieve near-optimal performance and facilitate the trade-offs between energy consumption, accuracy, and solution complexity. Simulation results show that the proposed approaches reduce the energy consumption significantly while executing multiple learning tasks compared to recent state-ofthe-art methods.

[1]  Jie Xu,et al.  Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design , 2020, J. Commun. Inf. Networks.

[2]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[3]  Geyong Min,et al.  Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT , 2020, IEEE Internet of Things Journal.

[4]  Edward Burnell,et al.  GPkit: A Human-Centered Approach to Convex Optimization in Engineering Design , 2020, CHI.

[5]  Gongxian Xu,et al.  Global optimization of signomial geometric programming problems , 2014, Eur. J. Oper. Res..

[6]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[7]  Jun Zhang,et al.  Edge-Assisted Hierarchical Federated Learning with Non-IID Data , 2019, ArXiv.

[8]  Peiping Shen,et al.  Global optimization of signomial geometric programming using linear relaxation , 2004, Appl. Math. Comput..

[9]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[10]  Jie Xu,et al.  D2D-Enabled Data Sharing for Distributed Machine Learning at Wireless Network Edge , 2020, IEEE Wireless Communications Letters.

[11]  Canh Dinh,et al.  Federated Learning Over Wireless Networks: Convergence Analysis and Resource Allocation , 2019, IEEE/ACM Transactions on Networking.

[12]  Friedrich Eisenbrand,et al.  Fast Integer Programming in Fixed Dimension , 2003, ESA.

[13]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[14]  Sameh Sorour,et al.  Optimal Task Allocation for Mobile Edge Learning with Global Training Time Constraints , 2020, 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC).

[15]  Hongyue WANG,et al.  Log-transformation and its implications for data analysis , 2014, Shanghai archives of psychiatry.

[16]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[17]  Sameh Sorour,et al.  Task Allocation for Mobile Federated and Offloaded Learning with Energy and Delay Constraints , 2020, 2020 IEEE International Conference on Communications Workshops (ICC Workshops).

[18]  Mohsen Guizani,et al.  Optimal User-Edge Assignment in Hierarchical Federated Learning Based on Statistical Properties and Network Topology Constraints , 2022, IEEE Transactions on Network Science and Engineering.

[19]  Klaus-Robert Müller,et al.  Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Stephen J. Wright,et al.  Primal-Dual Interior-Point Methods , 1997 .

[21]  Christodoulos A. Floudas,et al.  Deterministic global optimization - theory, methods and applications , 2010, Nonconvex optimization and its applications.

[22]  Kaibin Huang,et al.  Towards an Intelligent Edge: Wireless Communication Meets Machine Learning , 2018, ArXiv.

[23]  Sameh Sorour,et al.  Adaptive Task Allocation for Mobile Edge Learning , 2018, 2019 IEEE Wireless Communications and Networking Conference Workshop (WCNCW).

[24]  Mykel J. Kochenderfer,et al.  Algorithms for Optimization , 2019 .

[25]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[26]  Hao Wang,et al.  Optimizing Federated Learning on Non-IID Data with Reinforcement Learning , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[27]  Qiong Wu,et al.  HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learning , 2020, IEEE Transactions on Wireless Communications.