Rebalancing shared mobility-on-demand systems: A reinforcement learning approach

Shared mobility-on-demand systems have very promising prospects in making urban transportation efficient and affordable. However, due to operational challenges among others, many mobility applications still remain niche products. This paper addresses rebalancing needs that are critical for effective fleet management in order to offset the inevitable imbalance of vehicle supply and travel demand. Specifically, we propose a reinforcement learning approach which adopts a deep Q network and adaptively moves idle vehicles to regain balance. This innovative model-free approach takes a very different perspective from the state-of-the-art network-based methods and is able to cope with large-scale shared systems in real time with partial or full data availability. We apply this approach to an agent-based simulator and test it on a London case study. Results show that, the proposed method outperforms the local anticipatory method by reducing the fleet size by 14% while inducing little extra vehicle distance traveled. The performance is close to the optimal solution yet the computational speed is 2.5 times faster. Collectively, the paper concludes that the proposed rebalancing approach is effective under various demand scenarios and will benefit both travelers and operators if implemented in shared mobility-on-demand systems.

[1]  Jinhua Zhao,et al.  Transit-oriented autonomous vehicle operation with integrated demand-supply interaction , 2018, Transportation Research Part C: Emerging Technologies.

[2]  Jonathan P. How,et al.  Dynamic arrival rate estimation for campus Mobility On Demand network graphs , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Emilio Frazzoli,et al.  Toward a Systematic Approach to the Design and Evaluation of Automated Mobility-on-Demand Systems: A Case Study in Singapore , 2014 .

[4]  Matthijs T. J. Spaan,et al.  Traffic flow optimization: A reinforcement learning approach , 2016, Eng. Appl. Artif. Intell..

[5]  Emilio Frazzoli,et al.  Robotic load balancing for mobility-on-demand systems , 2012, Int. J. Robotics Res..

[6]  Kara M. Kockelman,et al.  Dynamic ride-sharing and fleet sizing for a system of shared autonomous vehicles in Austin, Texas , 2018 .

[7]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[8]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[9]  Mauro Dell'Amico,et al.  The bike sharing rebalancing problem: Mathematical formulations and benchmark instances , 2014 .

[10]  Patrick Jaillet,et al.  Dynamic Repositioning to Reduce Lost Demand in Bike Sharing Systems , 2017, J. Artif. Intell. Res..

[11]  Ruimin Li,et al.  Dynamic Pricing in Shared Mobility on Demand Service , 2018, 1802.03559.

[12]  Kian Hsiang Low,et al.  Gaussian Process Decentralized Data Fusion and Active Sensing for Spatiotemporal Traffic Modeling and Prediction in Mobility-on-Demand Systems , 2015, IEEE Transactions on Automation Science and Engineering.

[13]  Nikolas Geroliminis,et al.  An optimization framework for the development of efficient one-way car-sharing systems , 2015, Eur. J. Oper. Res..

[14]  Hongmou Zhang,et al.  Embedding Autonomous Vehicle Sharing in Public Transit System: An Example of Last-Mile Problem , 2017 .

[15]  Jongtae Rhee,et al.  Performance Analysis of a Forecasting Relocation Model for One-Way Carsharing , 2017 .

[16]  Stéphane Bressan,et al.  Routing an Autonomous Taxi with Reinforcement Learning , 2016, CIKM.

[17]  Vijay Kumar,et al.  Privacy-preserving vehicle assignment for mobility-on-demand systems , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Emilio Frazzoli,et al.  Simulation Framework for Rebalancing of Autonomous Mobility on Demand Systems , 2016 .

[19]  Kara M. Kockelman,et al.  Dynamic Ride-Sharing and Optimal Fleet Sizing for a System of Shared Autonomous Vehicles , 2015 .

[20]  Emilio Frazzoli,et al.  Shared-Vehicle Mobility-on-Demand Systems: A Fleet Operator's Guide to Rebalancing Empty Vehicles , 2016 .

[21]  Marco Pavone,et al.  Control of robotic mobility-on-demand systems: A queueing-theoretical perspective , 2014, Int. J. Robotics Res..

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Emilio Frazzoli,et al.  On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment , 2017, Proceedings of the National Academy of Sciences.

[24]  M. Keith Chen,et al.  Dynamic Pricing in a Labor Market: Surge Pricing and Flexible Work on the Uber Platform , 2016, EC.