Routing an Autonomous Taxi with Reinforcement Learning

Singapore's vision of a Smart Nation encompasses the development of effective and efficient means of transportation. The government's target is to leverage new technologies to create services for a demand-driven intelligent transportation model including personal vehicles, public transport, and taxis. Singapore's government is strongly encouraging and supporting research and development of technologies for autonomous vehicles in general and autonomous taxis in particular. The design and implementation of intelligent routing algorithms is one of the keys to the deployment of autonomous taxis. In this paper we demonstrate that a reinforcement learning algorithm of the Q-learning family, based on a customized exploration and exploitation strategy, is able to learn optimal actions for the routing autonomous taxis in a real scenario at the scale of the city of Singapore with pick-up and drop-off events for a fleet of one thousand taxis.

[1]  Thomas G. Dietterich.  The MAXQ Method for Hierarchical Reinforcement Learning , 1998, ICML.

[2]  Sridhar Mahadevan,et al.  A multiagent reinforcement learning algorithm by dynamically merging markov decision processes , 2002, AAMAS '02.

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[5]  Sridhar Mahadevan,et al.  Learning to communicate and act using hierarchical reinforcement learning , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[6]  Michael L. Littman,et al.  A hierarchical approach to efficient reinforcement learning in deterministic domains , 2006, AAMAS '06.

[7]  Peter Stone,et al.  Generalized model learning for reinforcement learning in factored domains , 2009, AAMAS.

[8]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[9]  Xing Xie,et al.  T-Drive: Enhancing Driving Directions with Taxi Drivers' Intelligence , 2013, IEEE Transactions on Knowledge and Data Engineering.

[10]  Nicholas Jing Yuan,et al.  T-Finder: A Recommender System for Finding Passengers and Vacant Taxis , 2013, IEEE Transactions on Knowledge and Data Engineering.

[11]  Tan Cheon Kheong,et al.  Autonomous Vehicles, Next Stop: Singapore , 2014 .

[12]  Guannan Liu,et al.  A cost-effective recommender system for taxi drivers , 2014, KDD.

[13]  Reza Langari,et al.  Autonomous Vehicles , 2016, Science.