Deep Reinforcement Learning for Pre-caching and Task Allocation in Internet of Vehicles

With the development of Internet of Vehicles and 5G network, there is an increasing demand for services from vehicle users. Mobile edge computing offers a solution, that is, processing tasks on the edge server to improve user quality of experience (QoE). However, given the constant changes in the location of users on fast-moving vehicles, it remains a challenge on how to efficiently and stably transmit data. To address it, a method of pre-caching and task allocation based on deep reinforcement learning is proposed in this paper. The files requested by vehicle users are pre-cached on roadside units (RSUs), and transmission tasks are dynamically allocated to vehicle to vehicle (V2V) transmission and vehicle to roadside unit (V2R) transmission based on the speed of transmission. To be specific, pre-caching and task allocation are modeled as Markov decision processes (MDP). Then, Deep Deterministic Policy Gradient (DDPG) is applied to determine the optimal ratio of pre-caching and task allocation. The performance of the algorithm in different situations is analyzed through simulation and it is compared with other algorithms. It is found that DDPG can maximize the data reception rate of fast-moving vehicles, thereby improving the QoE of vehicle users.

[1]  Jiwei Huang,et al.  A Simulation-Based Optimization Approach for Reliability-Aware Service Composition in Edge Computing , 2020, IEEE Access.

[2]  Hyunggon Park,et al.  Reliable Data Dissemination Strategy based on Systematic Network Coding in V2I Networks , 2019, 2019 International Conference on Information and Communication Technology Convergence (ICTC).

[3]  Antonella Molinaro,et al.  Improving V2R connectivity to provide ITS applications in IEEE 802.11p/WAVE VANETs , 2011, 2011 18th International Conference on Telecommunications.

[4]  Hyuk Lim,et al.  Mobility Support for Vehicular Cloud Radio-Access-Networks with Edge Computing , 2018, 2018 IEEE 7th International Conference on Cloud Networking (CloudNet).

[5]  Jinia Konar,et al.  Comparison of Various Learning Rate Scheduling Techniques on Convolutional Neural Network , 2020, 2020 IEEE International Students' Conference on Electrical,Electronics and Computer Science (SCEECS).

[6]  Mohsen Guizani,et al.  Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications , 2015, IEEE Communications Surveys & Tutorials.

[7]  Jiwei Huang,et al.  A Multi‐queue Approach of Energy Efficient Task Scheduling for Sensor Hubs , 2020 .

[8]  Ying Chen,et al.  TOFFEE: Task Offloading and Frequency Scaling for Energy Efficiency of Mobile Devices in Mobile Edge Computing , 2019, IEEE Transactions on Cloud Computing.

[9]  Wei Chen,et al.  GreenDelivery: proactive content caching and push with energy-harvesting-based small cells , 2015, IEEE Communications Magazine.

[10]  Ying Chen,et al.  Dynamic Computation Offloading in Edge Computing for Internet of Things , 2019, IEEE Internet of Things Journal.

[11]  Chunlin Chen,et al.  A novel DDPG method with prioritized experience replay , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[12]  Alagan Anpalagan,et al.  A Dynamic Priority Service Provision Scheme for Delay-Sensitive Applications in Fog Computing , 2018, 2018 29th Biennial Symposium on Communications (BSC).

[13]  Zhu Han,et al.  Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor–Critic Deep Reinforcement Learning , 2019, IEEE Internet of Things Journal.

[14]  Mehdi Bennis,et al.  Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning , 2018, IEEE Internet of Things Journal.

[15]  K. B. Letaief,et al.  A Survey on Mobile Edge Computing: The Communication Perspective , 2017, IEEE Communications Surveys & Tutorials.

[16]  Xianbin Wang,et al.  A Latency and Reliability Guaranteed Resource Allocation Scheme for LTE V2V Communication Systems , 2018, IEEE Transactions on Wireless Communications.

[17]  Silvio Savarese,et al.  Learning to Track: Online Multi-object Tracking by Decision Making , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  T Praveena,et al.  Distributed Deep Reinforcement Learning using TensorFlow , 2017, 2017 International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC).

[19]  Ying Chen,et al.  Energy Efficient Dynamic Offloading in Mobile Edge Computing for Internet of Things , 2019, IEEE Transactions on Cloud Computing.