Deep Reinforcement Learning based Resource Allocation in Low Latency Edge Computing Networks

In this paper, we investigate strategies for the allocation of computational resources using deep reinforcement learning in mobile edge computing networks that operate with finite blocklength codes to support low latency communications. The end-to-end (E2E) reliability of the service is addressed, while both the delay violation probability and the decoding error probability are taken into account. By employing a deep reinforcement learning method, namely deep Q-learning, we design an intelligent agent at the edge computing node to develop a real-time adaptive policy for computational resource allocation for offloaded tasks of multiple users in order to improve the average E2E reliability. Via simulations, we show that under different task arrival rates, the realized policy serves to increase the task number that decreases the delay violation rate while guaranteeing an acceptable level of decoding error probability. Moreover, we show that the proposed deep reinforcement learning approach outperforms the random and equal scheduling benchmarks.

[1]  James Gross,et al.  Optimal Scheduling of Reliability-Constrained Relaying System Under Outdated CSI in the Finite Blocklength Regime , 2016, IEEE Transactions on Vehicular Technology.

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  H. Vincent Poor,et al.  Channel Coding Rate in the Finite Blocklength Regime , 2010, IEEE Transactions on Information Theory.

[4]  Xinyu Yang,et al.  A Survey on the Edge Computing for the Internet of Things , 2018, IEEE Access.

[5]  Khaled Ben Letaief,et al.  Power-Delay Tradeoff in Multi-User Mobile-Edge Computing Systems , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  H. Vincent Poor,et al.  Latency and Reliability-Aware Task Offloading and Resource Allocation for Mobile Edge Computing , 2017, 2017 IEEE Globecom Workshops (GC Wkshps).

[8]  Mustafa Cenk Gursoy,et al.  Wireless Throughput and Energy Efficiency With Random Arrivals and Statistical Queuing Constraints , 2015, IEEE Transactions on Information Theory.

[9]  Raja Lavanya,et al.  Fog Computing and Its Role in the Internet of Things , 2019, Advances in Computer and Electrical Engineering.

[10]  Yulin Hu,et al.  Relaying-Enabled Ultra-Reliable Low-Latency Communications in 5G , 2018, IEEE Network.

[11]  Yan Zhang,et al.  Mobile Edge Computing: A Survey , 2018, IEEE Internet of Things Journal.

[12]  Jeongho Kwak,et al.  DREAM: Dynamic Resource and Task Allocation for Energy Minimization in Mobile Cloud Systems , 2015, IEEE Journal on Selected Areas in Communications.

[13]  W. Marsden I and J , 2012 .

[14]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[15]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.