Deep reinforcement learning based task scheduling scheme in mobile edge computing network

Mobile edge computing is a new distributed computing paradigm which brings computation and data storage closer to the location where it is needed, to improve response times and save bandwidth in the dynamic mobile networking environment. Despite the improvements in network technology, data centers cannot always guarantee acceptable transfer rates and response times, which could be a critical requirement for many applications. The aim of mobile edge computing is to move the computation away from data centers towards the edge of the network, exploiting smart objects, mobile phones or network gateways to perform tasks and provide services on behalf of the cloud. In this paper, we design a task offloading scheme in the mobile edge network to handle the task distribution, offloading and management by applying deep reinforcement learning. Specifically, we formulate the task offloading problem as a multi-agent reinforcement learning problem. The decision-making process of each agent is modeled as a Markov decision process and deep Q-learning approach is applied to deal with the large scale of states and actions. To evaluate the performance of our proposed scheme, we develop a simulation environment for the mobile edge computing scenario. Our preliminary evaluation results with a simplified multi-armed bandit model indicate that our proposed solution can provide lower latency for the computational intensive tasks in mobile edge network, and outperforms than naïve task offloading method.

[1]  Jiannong Cao,et al.  Heuristic offloading of concurrent tasks for computation-intensive applications in mobile cloud computing , 2014, 2014 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[2]  Guohong Cao,et al.  Energy-Efficient Computation Offloading in Cellular Networks , 2015, 2015 IEEE 23rd International Conference on Network Protocols (ICNP).

[3]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[4]  Zhisheng Niu,et al.  A Cooperative Scheduling Scheme of Local Cloud and Internet Cloud for Delay-Aware Mobile Cloud Computing , 2015, 2015 IEEE Globecom Workshops (GC Wkshps).

[5]  Doina Precup,et al.  Algorithms for multi-armed bandit problems , 2014, ArXiv.

[6]  Jiabin Wang,et al.  A Survey on Mobile Edge Computing: Focusing on Service Adoption and Provision , 2018, Wirel. Commun. Mob. Comput..

[7]  Bhaskar Krishnamachari,et al.  Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks , 2018, IEEE Transactions on Cognitive Communications and Networking.

[8]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[9]  Shimon Whiteson,et al.  Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning , 2017, ICML.

[10]  Peter Auer,et al.  The Nonstochastic Multiarmed Bandit Problem , 2002, SIAM J. Comput..

[11]  Mehryar Mohri,et al.  Multi-armed Bandit Algorithms and Empirical Evaluation , 2005, ECML.

[12]  Weifa Liang,et al.  Online Algorithms for Location-Aware Task Offloading in Two-Tiered Mobile Cloud Environments , 2014, 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing.

[13]  Carlos A. Varela,et al.  Light-weight adaptive task offloading from smartphones to nearby computational resources , 2011, RACS.

[14]  Min Chen,et al.  Task Offloading for Mobile Edge Computing in Software Defined Ultra-Dense Network , 2018, IEEE Journal on Selected Areas in Communications.

[15]  Shimon Whiteson,et al.  OFFER: Off-Environment Reinforcement Learning , 2017, AAAI.

[16]  Tao Jiang,et al.  Optimal radio resource allocation for mobile task offloading in cellular networks , 2014, IEEE Network.