QGrid: Q-learning based routing protocol for vehicular ad hoc networks

In Vehicular Ad Hoc Networks (VANETs), moving vehicles are considered as mobile nodes in the network and they are connected to each other via wireless links when they are within the communication radius of each other. Efficient message delivery in VANETs is still a very challenging research issue. In this paper, a Q-learning based routing protocol (i.e., QGrid) is introduced to help to improve the message delivery from mobile vehicles to a specific location. QGrid considers both macroscopic and microscopic aspects when making the routing decision, while the traditional routing methods focus on computing meeting information between different vehicles. QGrid divides the region into different grids. The macroscopic aspect determines the optimal next-hop grid and the microscopic aspect determines the specific vehicle in the optimal next-hop grid to be selected as next-hop vehicle. QGrid computes the Q-values of different movements between neighboring grids for a given destination via Q-learning. Each vehicle stores Q-value table learned offline, then selects optimal next-hop grid by querying Q-value table. Inside the selected next-hop grid, we either greedily select the nearest neighboring vehicle to the destination or select the neighboring vehicle with highest probability of moving to the optimal next-hop grid predicted by the two-order Markov chain. The performance of QGrid is evaluated by using real life trajectory GPS data of Shanghai taxies. Simulation comparison among QGrid and other existing position-based routing protocols confirms the advantages of proposed QGrid routing protocol for VANETs.

[1]  Yuh-Shyan Chen,et al.  Routing Protocols in Vehicular Ad Hoc Networks: A Survey and Future Perspectives , 2010, J. Inf. Sci. Eng..

[2]  Yan Huang,et al.  Intersection-based forwarding protocol for vehicular ad hoc networks , 2015, Telecommunication Systems.

[3]  Celimuge Wu,et al.  Distributed Reinforcement Learning Approach for Vehicular Ad Hoc Networks , 2010, IEICE Trans. Commun..

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Celimuge Wu,et al.  Routing in VANETs: A fuzzy constraint Q-Learning approach , 2012, 2012 IEEE Global Communications Conference (GLOBECOM).

[6]  Minyi Guo,et al.  Adaptive Forwarding Delay Control for VANET Data Aggregation , 2012, IEEE Transactions on Parallel and Distributed Systems.

[7]  Brad Karp,et al.  GPSR: greedy perimeter stateless routing for wireless networks , 2000, MobiCom '00.

[8]  Mianxiong Dong,et al.  ZOOM: Scaling the mobility for fast opportunistic forwarding in vehicular networks , 2013, 2013 Proceedings IEEE INFOCOM.

[9]  Victor C. M. Leung,et al.  Position-Based Directional Vehicular Routing , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[10]  Lei Zhao,et al.  Hybrid Position-Based and DTN Forwarding in Vehicular Ad Hoc Networks , 2012, 2012 IEEE Vehicular Technology Conference (VTC Fall).

[11]  Jianping Pan,et al.  GeoMob: A mobility-aware geocast scheme in metropolitans via taxicabs and buses , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[12]  Shinji Kusumoto,et al.  GVGrid: A QoS Routing Protocol for Vehicular Ad Hoc Networks , 2006, 200614th IEEE International Workshop on Quality of Service.

[13]  Lei Zhao,et al.  Hybrid Position-Based and DTN Forwarding for Vehicular Sensor Networks , 2012, Int. J. Distributed Sens. Networks.

[14]  Sagar Naik,et al.  Exploiting temporal dependency for opportunistic forwarding in urban vehicular networks , 2011, 2011 Proceedings IEEE INFOCOM.

[15]  Martin Mauve,et al.  Geographic routing in city scenarios , 2005, MOCO.

[16]  Shiow-Fen Hwang,et al.  HarpiaGrid: A Geography-Aware Grid-based Routing Protocol for Vehicular Ad Hoc Networks , 2010, J. Inf. Sci. Eng..

[17]  Yu Wang,et al.  Routing in vehicular ad hoc networks: A survey , 2007, IEEE Vehicular Technology Magazine.

[18]  Yan Huang,et al.  Multicast capacity analysis for social-proximity urban bus-assisted VANETs , 2013, 2013 IEEE International Conference on Communications (ICC).