Minimizing Age-of-Information for Fog Computing-supported Vehicular Networks with Deep Q-learning

Connected vehicular network is one of the key enablers for next generation cloud/fog-supported autonomous driving vehicles. Most connected vehicular applications require frequent status updates and Age of Information (AoI) is a more relevant metric to evaluate the performance of wireless links between vehicles and cloud/fog servers. This paper introduces a novel proactive and data-driven approach to optimize the driving route with a main objective of guaranteeing the confidence of AoI. In particular, we report a study on three month measurements of a multi-vehicle campus shuttle system connected to cloud/fog servers via a commercial LTE network. We establish empirical models for AoI in connected vehicles and investigate the impact of major factors on the performance of AoI. We also propose a Deep Q-Learning Netwrok (DQN)-based algorithm to decide the optimal driving route for each connected vehicle with maximized confidence level. Numerical results show that the proposed approach can lead to a significant improvement on the AoI confidence for various types of services supported.

[1]  Anthony Ephremides,et al.  Effect of Message Transmission Path Diversity on Status Age , 2016, IEEE Transactions on Information Theory.

[2]  Csaba Szepesvári,et al.  The Asymptotic Convergence-Rate of Q-learning , 1997, NIPS.

[3]  Roy D. Yates,et al.  Update or wait: How to keep your data fresh , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[4]  Eryk Dutkiewicz,et al.  Optimal and Fast Real-Time Resource Slicing With Deep Dueling Neural Networks , 2019, IEEE Journal on Selected Areas in Communications.

[5]  Sanjit Krishnan Kaul,et al.  Minimizing age of information in vehicular networks , 2011, 2011 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[6]  Marwan Krunz,et al.  Distributed Optimization for Energy-Efficient Fog Computing in the Tactile Internet , 2018, IEEE Journal on Selected Areas in Communications.

[7]  Xavier Masip-Bruin,et al.  What is a Fog Node A Tutorial on Current Concepts towards a Common Definition , 2016, ArXiv.

[8]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning for Resource Allocation in V2V Communications , 2017, 2018 IEEE International Conference on Communications (ICC).

[9]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[10]  Cheng-Xiang Wang,et al.  Spatial Spectrum and Energy Efficiency of Random Cellular Networks , 2015, IEEE Transactions on Communications.

[11]  Marwan Krunz,et al.  Driving in the Fog: Latency Measurement, Modeling, and Optimization of LTE-based Fog Computing for Smart Vehicles , 2019, 2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON).

[12]  Hiroyuki Koga,et al.  Analysis of fog model considering computing and communication latency in 5G cellular networks , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[13]  Marco Fiore,et al.  Temporal Reachability in Vehicular Networks , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[14]  Xiaohu Ge,et al.  User Mobility Evaluation for 5G Small Cell Networks Based on Individual Mobility Model , 2015, IEEE Journal on Selected Areas in Communications.

[15]  Rajkumar Buyya,et al.  Quality of Experience (QoE)-aware placement of applications in Fog computing environments , 2019, J. Parallel Distributed Comput..