MDP-Based Task Offloading for Vehicular Edge Computing Under Certain and Uncertain Transition Probabilities

Low latency/delay is one of the most critical requirements for the application of vehicular networks. However, frequent real-time information update caused by vehicles high mobility is liable to aggravate the delay. Meanwhile, the task migration between different vehicular edge computing (VEC) servers results in an amount of delay if the computing cannot be completed before the vehicle moves out of the coverage of the current VEC server. In this paper, the problem is concluded as when and to whom to offload the task for VEC, which is formulated as a finite horizon Markov decision process (MDP) to minimize the delay with respect to the communication, computing, handover and migration. Through characterizing the time-space correlation of vehicles mobility, the curse of dimensionality problem in MDP is resolved. Meanwhile, a general expression of the transition probabilities is derived. On this basis, the specific results of highway, 2-D street and real-data scenarios are provided as well. For practical implementation considerations, the transition probabilities are commonly uncertain primarily due to random driver behavior, inaccurate sample data and complex path environment. Under this uncertain environment,a robust time-aware MDP-based task offloading algorithm (RTMDP) is proposed, which has been proved to perform well even under the high uncertain transition probabilities by simulation results.

[1]  Hossam S. Hassanein,et al.  Vehicle as a resource (VaaR) , 2014, IEEE Network.

[2]  Mehdi Bennis,et al.  Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning , 2018, IEEE Internet of Things Journal.

[3]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[4]  Xuemin Shen,et al.  An Optimized and Distributed Data Packet Forwarding Scheme in LTE/LTE-A Networks , 2016, IEEE Transactions on Vehicular Technology.

[5]  Dong In Kim,et al.  Optimal Energy Management Policy of Mobile Energy Gateway , 2016, IEEE Transactions on Vehicular Technology.

[6]  Khaled Ben Letaief,et al.  Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices , 2016, IEEE Journal on Selected Areas in Communications.

[7]  Dusit Niyato,et al.  Offloading in Mobile Cloudlet Systems with Intermittent Connectivity , 2015, IEEE Transactions on Mobile Computing.

[8]  Vincent W. S. Wong,et al.  DORA: Dynamic Optimal Random Access for Vehicle-to-Roadside Communications , 2012, IEEE Journal on Selected Areas in Communications.

[9]  Qimei Cui,et al.  Multi-Slot Coverage Probability and SINR-Based Handover Rate Analysis for Mobile User in Hetnet , 2018, IEEE Access.

[10]  Honggang Wang,et al.  Knowledge-Centric Edge Computing Based on Virtualized D2D Communication Systems , 2018, IEEE Communications Magazine.

[11]  K. Sohraby,et al.  A Generalized Random Mobility Model for Wireless Ad Hoc Networks and Its Analysis: One-Dimensional Case , 2007, IEEE/ACM Transactions on Networking.

[12]  Hwee Pink Tan,et al.  QoS-aware data transmission and wireless energy transfer: Performance modeling and optimization , 2014, 2014 IEEE Wireless Communications and Networking Conference (WCNC).

[13]  Lei Sun,et al.  Exploring device-to-device communication for mobile cloud computing , 2014, 2014 IEEE International Conference on Communications (ICC).

[14]  Peng Cheng,et al.  Cooperative data dissemination in cellular-VANET heterogeneous wireless networks , 2012, 2012 4th International High Speed Intelligent Communication Forum.

[15]  Ke Zhang,et al.  Delay constrained offloading for Mobile Edge Computing in cloud-enabled vehicular networks , 2016, 2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM).

[16]  Qi Zhang,et al.  Offloading Schemes in Mobile Edge Computing for Ultra-Reliable Low Latency Communications , 2018, IEEE Access.

[17]  Wei Ni,et al.  Stochastic Online Learning for Mobile Edge Computing: Learning from Changes , 2019, IEEE Communications Magazine.

[18]  Depeng Jin,et al.  Vehicular Fog Computing: A Viewpoint of Vehicles as the Infrastructures , 2016, IEEE Transactions on Vehicular Technology.

[19]  Markus Rupp,et al.  Signal Processing Challenges in Cellular-Assisted Vehicular Communications: Efforts and developments within 3GPP LTE and beyond , 2017, IEEE Signal Processing Magazine.

[20]  Wei Ni,et al.  Optimal Schedule of Mobile Edge Computing for Internet of Things Using Partial Information , 2017, IEEE Journal on Selected Areas in Communications.

[21]  Laurent El Ghaoui,et al.  Robust Solutions to Markov Decision Problems with Uncertain Transition Matrices , 2005 .

[22]  Ke Zhang,et al.  Mobile-Edge Computing for Vehicular Networks: A Promising Network Paradigm with Predictive Off-Loading , 2017, IEEE Veh. Technol. Mag..

[23]  Lin Tian,et al.  Mobile Edge Computing-Assisted Admission Control in Vehicular Networks: The Convergence of Communication and Computation , 2019, IEEE Vehicular Technology Magazine.

[24]  Xiaoli Chu,et al.  Computation Offloading and Resource Allocation in Vehicular Networks Based on Dual-Side Cost Minimization , 2019, IEEE Transactions on Vehicular Technology.

[25]  Xuefei Zhang,et al.  Delay-Optimal Temporal-Spatial Computation Offloading Schemes for Vehicular Edge Computing Systems , 2019, 2019 IEEE Wireless Communications and Networking Conference (WCNC).

[26]  Sheng Chen,et al.  A Markov Jump Process Model for Urban Vehicular Mobility: Modeling and Applications , 2014, IEEE Transactions on Mobile Computing.

[27]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[28]  Ying-Chang Liang,et al.  The SMART Handoff Policy for Millimeter Wave Heterogeneous Cellular Networks , 2018, IEEE Transactions on Mobile Computing.

[29]  Zhu Han,et al.  Optimal Wireless Energy Charging for Incentivized Content Transfer in Mobile Publish–Subscribe Networks , 2017, IEEE Transactions on Vehicular Technology.

[30]  Ke Zhang,et al.  Collaborative Task Offloading in Vehicular Edge Multi-Access Networks , 2018, IEEE Communications Magazine.

[31]  Min Chen,et al.  Mobility-Aware Caching and Computation Offloading in 5G Ultra-Dense Cellular Networks , 2016, Sensors.

[32]  Mohsen Guizani,et al.  On WiFi Offloading in Heterogeneous Networks: Various Incentives and Trade-Off Strategies , 2016, IEEE Communications Surveys & Tutorials.

[33]  Qimei Cui,et al.  The SIR Meta Distribution in Poisson Cellular Networks With Base Station Cooperation , 2018, IEEE Transactions on Communications.

[34]  Yan Zhang,et al.  Optimal delay constrained offloading for vehicular edge computing networks , 2017, 2017 IEEE International Conference on Communications (ICC).

[35]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[36]  Kaibin Huang,et al.  Live Prefetching for Mobile Computation Offloading , 2016, IEEE Transactions on Wireless Communications.

[37]  Ozan K. Tonguz,et al.  Modeling urban traffic: A cellular automata approach , 2009, IEEE Communications Magazine.

[38]  Man Hon Cheung,et al.  DAWN: Delay-Aware Wi-Fi Offloading and Network Selection , 2015, IEEE Journal on Selected Areas in Communications.

[39]  Sundeep Rangan,et al.  An MDP model for optimal handover decisions in mmWave cellular networks , 2015, 2016 European Conference on Networks and Communications (EuCNC).

[40]  Yan Zhang,et al.  A Reinforcement Learning-Based Data Storage Scheme for Vehicular Ad Hoc Networks , 2017, IEEE Transactions on Vehicular Technology.

[41]  Huan Zhou,et al.  V2V Data Offloading for Cellular Network Based on the Software Defined Network (SDN) Inside Mobile Edge Computing (MEC) Architecture , 2018, IEEE Access.