Learning-Based Offloading of Tasks with Diverse Delay Sensitivities for Mobile Edge Computing

The ever-evolving mobile applications need more and more computing resources to smooth user experience and sometimes meet delay requirements. Therefore, mobile devices (MDs) are gradually having difficulties to complete all tasks in time due to the limitations of computing power and battery life. To cope with this problem, mobile edge computing (MEC) systems were created to help with task processing for MDs at nearby edge servers. Existing works have been devoted to solving MEC task offloading problems, including those with simple delay constraints, but most of them neglect the coexistence of deadline-constrained and delay- sensitive tasks (i.e., the diverse delay sensitivities of tasks). In this paper, we propose an actor-critic based deep reinforcement learning (ADRL) model that takes the diverse delay sensitivities into account and offloads tasks adaptively to minimize the total penalty caused by deadline misses of deadline-constrained tasks and the lateness of delay-sensitive tasks. We train the ADRL model using a real data set that consists of the diverse delay sensitivities of tasks. Our simulation results show that the proposed solution outperforms several heuristic algorithms in terms of total penalty, and it also retains its performance gains under different system settings.

[1]  Ping Zhang,et al.  Stochastic Control of Computation Offloading to a Dynamic Helper , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Wenzhong Li,et al.  Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing , 2015, IEEE/ACM Transactions on Networking.

[4]  Chen-Khong Tham,et al.  A deep reinforcement learning based offloading scheme in ad-hoc mobile clouds , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[5]  Yusheng Ji,et al.  HVC: A Hybrid Cloud Computing Framework in Vehicular Environments , 2017, 2017 5th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud).

[6]  Mehdi Bennis,et al.  Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning , 2018, IEEE Internet of Things Journal.

[7]  Shaolei Ren,et al.  Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing , 2017, IEEE Transactions on Cognitive Communications and Networking.

[8]  Yu Cheng,et al.  Online SLA-Aware Multi-Resource Allocation for Deadline Sensitive Jobs in Edge-Clouds , 2017, GLOBECOM 2017 - 2017 IEEE Global Communications Conference.

[9]  Yusheng Ji,et al.  AVE: Autonomous Vehicular Edge Computing Framework with ACO-Based Scheduling , 2017, IEEE Transactions on Vehicular Technology.

[10]  Wei Zhou,et al.  Computational offloading with delay and capacity constraints in mobile edge , 2017, 2017 IEEE International Conference on Communications (ICC).

[11]  Tiejun Lv,et al.  Deep reinforcement learning based computation offloading and resource allocation for MEC , 2018, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[12]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[13]  Kaibin Huang,et al.  Energy Efficient Mobile Cloud Computing Powered by Wireless Energy Transfer , 2015, IEEE Journal on Selected Areas in Communications.

[14]  Tarik Taleb,et al.  Mobile Edge Computing Potential in Making Cities Smarter , 2017, IEEE Communications Magazine.

[15]  Zdenek Becvar,et al.  Mobile Edge Computing: A Survey on Architecture and Computation Offloading , 2017, IEEE Communications Surveys & Tutorials.

[16]  Xu Chen,et al.  Chimera: An Energy-Efficient and Deadline-Aware Hybrid Edge Computing Framework for Vehicular Crowdsensing Applications , 2019, IEEE Internet of Things Journal.

[17]  Xin Wang,et al.  Computation offloading for mobile edge computing: A deep learning approach , 2017, 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[18]  Sergio Barbarossa,et al.  Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing , 2014, IEEE Transactions on Signal and Information Processing over Networks.

[19]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[20]  Victor C. M. Leung,et al.  Energy Efficient Computation Offloading for Multi-Access MEC Enabled Small Cell Networks , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Anil A. Bharath,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.