Distributed Edge Computing Resource Allocation Algorithm Based on DRL in LTE Access Network

With the advent of IoT, as an extension of the traditional cloud-centric computing model, the edge computing model solves the network channel congestion caused by the increasing number of IoT devices and satisfies theirs low latency requirements. Since the edge network needs to adopt heterogeneous structure to meet multiple access requirements during construction, how to properly allocate limited edge resources to process computing requests from mobile edge terminals becomes a challenge under heterogeneous networks. In recent years, reinforcement learning shine in various types of decision-making problems, it has brought new ideas to solve complex decision problems. In this paper, we use reinforcement learning to solve the resource allocation problem of heterogeneous edge server (ES Edge Server) collaborative computing. In this model, we consider the multitasking and the heterogeneity and mobility of edge devices, abstracting the problem into Markov Decision Processes (MDPs) based on the real-time state of the network and the attributes of the task, using based on Actor Critic and Policy Gradient’s DDPG (Deep Deterministic Policy Gradient) make resource allocation optimization decisions. The simulation results show that compared with the DQN-based algorithm, the DDPG-based algorithm has better simulation results in the experiment.

[1]  Mehdi Bennis,et al.  Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning , 2018, IEEE Internet of Things Journal.

[2]  Roberto Di Pietro,et al.  Edge Computing Perspectives: Architectures, Technologies, and Open Security Issues , 2019, 2019 IEEE International Conference on Edge Computing (EDGE).

[3]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[4]  Gang Feng,et al.  iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks , 2019, IEEE Internet of Things Journal.

[5]  Di Chen,et al.  Weighted Max-Min Fairness oriented load-balancing and clustering for multicast Cache-Enabled F-RAN , 2016, 2016 9th International Symposium on Turbo Codes and Iterative Information Processing (ISTC).

[6]  Xiang Liu,et al.  Latency-Optimal Task Offloading for Mobile-Edge Computing System in 5G Heterogeneous Networks , 2018, 2018 IEEE 87th Vehicular Technology Conference (VTC Spring).

[7]  Gwendal Simon,et al.  A hybrid edge-cloud architecture for reducing on-demand gaming latency , 2014, Multimedia Systems.

[8]  Norio Shiratori,et al.  A Multi-agent Based Flexible IoT Edge Computing Architecture and Application to ITS , 2019, J. Commun..

[9]  Antonio Puliafito,et al.  A Deep Reinforcement Learning Approach For Data Migration in Multi-Access Edge Computing , 2018, 2018 ITU Kaleidoscope: Machine Learning for a 5G Future (ITU K).

[10]  Ebtesam J. Alqahtani,et al.  Securing Communication between Fog Computing and IoT Using Constrained Application Protocol (CoAP): A Survey , 2020, J. Commun..

[11]  Zibin Zheng,et al.  Online Deep Reinforcement Learning for Computation Offloading in Blockchain-Empowered Mobile Edge Computing , 2019, IEEE Transactions on Vehicular Technology.

[12]  Richard D. Gitlin,et al.  A Clustering Algorithm That Maximizes Throughput in 5G Heterogeneous F-RAN Networks , 2018, 2018 IEEE International Conference on Communications (ICC).

[13]  Zhi Zhang,et al.  Energy-Aware Mobile Edge Computation Offloading for IoT Over Heterogenous Networks , 2019, IEEE Access.

[14]  Heiko Ludwig,et al.  Zenith: Utility-Aware Resource Allocation for Edge Computing , 2017, 2017 IEEE International Conference on Edge Computing (EDGE).

[15]  Ruchuan Wang,et al.  Requirement-Oriented Privacy Protection Analysis Architecture in Cloud Computing , 2015, J. Commun..

[16]  Yang Liu,et al.  A Parallel Task Scheduling Optimization Algorithm Based on Clonal Operator in Green Cloud Computing , 2016, J. Commun..

[17]  Xinyu Yang,et al.  A Survey on the Edge Computing for the Internet of Things , 2018, IEEE Access.

[18]  Ammar Muthanna,et al.  Study of fog computing structure , 2018, 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus).