A Joint Service Migration and Mobility Optimization Approach for Vehicular Edge Computing

The vehicular edge computing is considered an enabling technology for intelligent and connected vehicles since the optimization of communication and computing on edge has a significant impact on driving safety and efficiency. In this paper, with the road traffic assignment to “proactively” reshape the spatiotemporal distribution of resource demands, we investigate the joint service migration and mobility optimization problem for vehicular edge computing. The goal is to meet the service delay requirements of vehicular edge computing with minimum migration cost and travel time. As service migration and mobility optimization are coupled, the joint scheduling problem suffers from the curse of dimensionality, which cannot be solved in real time by centralized algorithms. To this end, a multi-agent deep reinforcement learning (MADRL) algorithm is proposed to maximize the composite utility of communication, computing, and route planning in a distributed way. In the MADRL algorithm, a two-branch convolution based deep Q-network is constructed to coordinate migration action and routing action. Extensive experimental results show that the proposed algorithm is scalable and substantially reduces service delay, migration cost and travel time as compared with the existing baselines.

[1]  Bart De Schutter,et al.  Hierarchical Game Theory for System-Optimal Control: Applications of Reverse Stackelberg Games in Regulating Marketing Channels and Traffic Routing , 2017, IEEE Control Systems.

[2]  Zhao Chen,et al.  Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach , 2018, EURASIP Journal on Wireless Communications and Networking.

[3]  Tianshu Chu,et al.  Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control , 2019, IEEE Transactions on Intelligent Transportation Systems.

[4]  Min Chen,et al.  Task Offloading for Mobile Edge Computing in Software Defined Ultra-Dense Network , 2018, IEEE Journal on Selected Areas in Communications.

[5]  Xuemin Shen,et al.  Toward Dynamic Link Utilization for Efficient Vehicular Edge Content Distribution , 2019, IEEE Transactions on Vehicular Technology.

[6]  Yanhua Zhang,et al.  Delay-Tolerant Data Traffic to Software-Defined Vehicular Networks With Mobile Edge Computing in Smart City , 2018, IEEE Transactions on Vehicular Technology.

[7]  Dongning Guo,et al.  Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks , 2018, IEEE Journal on Selected Areas in Communications.

[8]  Feng Lyu,et al.  Toward Collision-Free and Efficient Coordination for Automated Vehicles at Unsignalized Intersection , 2019, IEEE Internet of Things Journal.

[9]  Wenchao Xu,et al.  Big Data Driven Vehicular Networks , 2018, IEEE Network.

[10]  Nei Kato,et al.  Future Intelligent and Secure Vehicular Network Toward 6G: Machine-Learning Approaches , 2020, Proceedings of the IEEE.

[11]  Kai Lin,et al.  Vehicle Route Selection Based on Game Evolution in Social Internet of Vehicles , 2018, IEEE Internet of Things Journal.

[12]  Mohsen Guizani,et al.  Performance Evaluation of IEEE 802.15.4 Nonbeacon-Enabled Mode for Internet of Vehicles , 2015, IEEE Transactions on Intelligent Transportation Systems.

[13]  Wei Wang,et al.  Evolutionary V2X Technologies Toward the Internet of Vehicles: Challenges and Opportunities , 2020, Proceedings of the IEEE.

[14]  Nei Kato,et al.  Machine Learning Meets Computation and Communication Control in Evolving Edge and Cloud: Challenges and Future Perspective , 2020, IEEE Communications Surveys & Tutorials.

[15]  Hado van Hasselt,et al.  Double Q-learning , 2010, NIPS.

[16]  Xuemin Shen,et al.  Toward Efficient Content Delivery for Automated Driving Services: An Edge Computing Solution , 2018, IEEE Network.

[17]  Jie Xu,et al.  Computation Peer Offloading for Energy-Constrained Mobile Edge Computing in Small-Cell Networks , 2017, IEEE/ACM Transactions on Networking.

[18]  S. Wong,et al.  Biased standard error estimations in transport model calibration due to heteroscedasticity arising from the variability of linear data projection , 2016 .

[19]  Jie Xu,et al.  EMM: Energy-Aware Mobility Management for Mobile Edge Computing in Ultra Dense Networks , 2017, IEEE Journal on Selected Areas in Communications.

[20]  Monireh Abdoos,et al.  Traffic light control in non-stationary environments based on multi agent Q-learning , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[21]  Weihua Zhuang,et al.  SDN/NFV-Empowered Future IoV With Enhanced Communication, Computing, and Caching , 2020, Proceedings of the IEEE.

[22]  Tarik Taleb,et al.  Follow-Me Cloud: When Cloud Services Follow Mobile Users , 2019, IEEE Transactions on Cloud Computing.

[23]  Yanning Zhang,et al.  Smart and Resilient EV Charging in SDN-Enhanced Vehicular Edge Computing Networks , 2020, IEEE Journal on Selected Areas in Communications.

[24]  Jeffrey G. Andrews,et al.  Reinforcement Learning for Self Organization and Power Control of Two-Tier Heterogeneous Networks , 2018, IEEE Transactions on Wireless Communications.

[25]  Nan Cheng,et al.  Software-Defined Cooperative Data Sharing in Edge Computing Assisted 5G-VANET , 2021, IEEE Transactions on Mobile Computing.

[26]  Ning Lu,et al.  Soft-defined heterogeneous vehicular network: architecture and challenges , 2015, IEEE Network.

[27]  Fangchun Yang,et al.  A Traffic Prediction Enabled Double Rewarded Value Iteration Network for Route Planning , 2019, IEEE Transactions on Vehicular Technology.

[28]  Weihua Zhuang,et al.  AI-Assisted Network-Slicing Based Next-Generation Wireless Networks , 2020, IEEE Open Journal of Vehicular Technology.

[29]  Jie Zhang,et al.  Maximizing the Probability of Arriving on Time: A Practical Q-Learning Method , 2017, AAAI.

[30]  Yuan Wu,et al.  NOMA-Assisted Multi-Access Mobile Edge Computing: A Joint Optimization of Computation Offloading and Time Allocation , 2018, IEEE Transactions on Vehicular Technology.

[31]  Weihua Zhuang,et al.  End-to-End Quality of Service in 5G Networks: Examining the Effectiveness of a Network Slicing Framework , 2018, IEEE Vehicular Technology Magazine.

[32]  Xu Chen,et al.  Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computing , 2018, 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS).

[33]  Qi Qi,et al.  Performance Optimization for Blockchain-Enabled Distributed Network Function Virtualization Management and Orchestration , 2020, IEEE Transactions on Vehicular Technology.

[34]  Wei Li,et al.  A Dynamic Service Migration Mechanism in Edge Cognitive Computing , 2018, ACM Trans. Internet Techn..

[35]  Nan Zhao,et al.  Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[36]  Lenan Wu,et al.  Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches , 2019, IEEE Transactions on Wireless Communications.

[37]  Rose Qingyang Hu,et al.  Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning , 2018, IEEE Transactions on Vehicular Technology.