Optimization of lightweight task offloading strategy for mobile edge computing based on deep reinforcement learning

Abstract With the maturity of 5G technology and the popularity of intelligent terminal devices, the traditional cloud computing service model cannot deal with the explosive growth of business data quickly. Therefore, the purpose of mobile edge computing (MEC) is to effectively solve problems such as latency and network load. In this paper, deep reinforcement learning (DRL) is first proposed to solve the offloading problem of multiple service nodes for the cluster and multiple dependencies for mobile tasks in large-scale heterogeneous MEC. Then the paper uses the LSTM network layer and the candidate network set to improve the DQN algorithm in combination with the actual environment of the MEC. Finally, the task offloading problem is simulated by using iFogSim and Google Cluster Trace. The simulation results show that the offloading strategy based on the improved IDRQN algorithm has better performance in energy consumption, load balancing, latency and average execution time than other algorithms.

[1]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[2]  Lei Cao,et al.  Deep Reinforcement Learning with Sarsa and Q-Learning: A Hybrid Approach , 2018, IEICE Trans. Inf. Syst..

[3]  Rajkumar Buyya,et al.  iFogSim: A toolkit for modeling and simulation of resource management techniques in the Internet of Things, Edge and Fog computing environments , 2016, Softw. Pract. Exp..

[4]  Hamid Reza Arkian,et al.  MIST: Fog-based data analytics scheme with cost-efficient resource provisioning for IoT crowdsensing applications , 2017, J. Netw. Comput. Appl..

[5]  Adlen Ksentini,et al.  On Using Edge Computing for Computation Offloading in Mobile Network , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[6]  Khaled Ben Letaief,et al.  Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices , 2016, IEEE Journal on Selected Areas in Communications.

[7]  Zdenek Becvar,et al.  Mobile Edge Computing: A Survey on Architecture and Computation Offloading , 2017, IEEE Communications Surveys & Tutorials.

[8]  H. Vincent Poor,et al.  Latency and Reliability-Aware Task Offloading and Resource Allocation for Mobile Edge Computing , 2017, 2017 IEEE Globecom Workshops (GC Wkshps).

[9]  Wenzhong Li,et al.  Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing , 2015, IEEE/ACM Transactions on Networking.

[10]  Ke Zhang,et al.  Energy-Efficient Offloading for Mobile Edge Computing in 5G Heterogeneous Networks , 2016, IEEE Access.

[11]  Ruiying Li,et al.  Historical Best Q-Networks for Deep Reinforcement Learning , 2018, 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI).

[12]  Qianbin Chen,et al.  Computation Offloading and Resource Allocation in Wireless Cellular Networks With Mobile Edge Computing , 2017, IEEE Transactions on Wireless Communications.

[13]  Philipp Leitner,et al.  Optimized IoT service placement in the fog , 2017, Service Oriented Computing and Applications.

[14]  Marcin Andrychowicz,et al.  Hindsight Experience Replay , 2017, NIPS.

[15]  Kaibin Huang,et al.  Energy-Efficient Resource Allocation for Mobile-Edge Computation Offloading , 2016, IEEE Transactions on Wireless Communications.

[16]  Yunlong Cai,et al.  Partial Offloading for Latency Minimization in Mobile-Edge Computing , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[17]  Carlos Juiz,et al.  A lightweight decentralized service placement policy for performance optimization in fog computing , 2018, Journal of Ambient Intelligence and Humanized Computing.

[18]  Tony Q. S. Quek,et al.  Offloading in Mobile Edge Computing: Task Allocation and Computational Frequency Scaling , 2017, IEEE Transactions on Communications.

[19]  Yuan Wu,et al.  Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing , 2019, Digit. Commun. Networks.

[20]  Shuihua Wang,et al.  Cerebral Micro-Bleeding Detection Based on Densely Connected Neural Network , 2019, Front. Neurosci..

[21]  Mainak Adhikari,et al.  Heuristic-based load-balancing algorithm for IaaS cloud , 2018, Future Gener. Comput. Syst..

[22]  Yong Xiang,et al.  Cost Efficient Resource Management in Fog Computing Supported Medical Cyber-Physical System , 2017, IEEE Transactions on Emerging Topics in Computing.

[23]  Li Zhou,et al.  Energy-Latency Tradeoff for Energy-Aware Offloading in Mobile Edge Computing Networks , 2018, IEEE Internet of Things Journal.

[24]  Shipeng Xie,et al.  Alcoholism Identification Based on an AlexNet Transfer Learning Model , 2019, Front. Psychiatry.

[25]  Qi Zhang,et al.  Offloading Schemes in Mobile Edge Computing for Ultra-Reliable Low Latency Communications , 2018, IEEE Access.

[26]  Yunfeng Ai,et al.  Parallel reinforcement learning: a framework and case study , 2018, IEEE/CAA Journal of Automatica Sinica.

[27]  Jun Zhang,et al.  Stochastic Joint Radio and Computational Resource Management for Multi-User Mobile-Edge Computing Systems , 2017, IEEE Transactions on Wireless Communications.

[28]  Jie Wu,et al.  A Multi-objective Biogeography-Based Optimization for Virtual Machine Placement , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[29]  Laetitia Lepetit,et al.  Bank Insolvency Risk and Time-Varying Z-Score Measures , 2013 .

[30]  Xu Han,et al.  Cost Aware Service Placement and Load Dispatching in Mobile Cloud Systems , 2016, IEEE Transactions on Computers.