Approximate Dynamic Programming for Planning a Ride-Sharing System using Autonomous Fleets of Electric Vehicles

Within a decade, almost every major auto company, along with fleet operators such as Uber, have announced plans to put autonomous vehicles on the road. At the same time, electric vehicles are quickly emerging as a next-generation technology that is cost effective, in addition to offering the benefits of reducing the carbon footprint. The combination of a centrally managed fleet of driverless vehicles, along with the operating characteristics of electric vehicles, is creating a transformative new technology that offers significant cost savings with high service levels. This problem involves a dispatch problem for assigning riders to cars, a surge pricing problem for deciding on the price per trip and a planning problem for deciding on the fleet size. We use approximate dynamic programming to develop high-quality operational dispatch strategies to determine which car is best for a particular trip, when a car should be recharged, and when it should be re-positioned to a different zone which offers a higher density of trips. We prove that the value functions are monotone in the battery and time dimensions and use hierarchical aggregation to get better estimates of the value functions with a small number of observations. Then, surge pricing is discussed using an adaptive learning approach to decide on the price for each trip. Finally, we discuss the fleet size problem which depends on the previous two problems.

[1]  Erik Brynjolfsson,et al.  Race against the machine : how the digital revolution is accelerating innovation, driving productivity, and irreversibly transforming employment and the economy , 2011 .

[2]  Zechun Hu,et al.  Decentralized Vehicle-to-Grid Control for Primary Frequency Regulation Considering Charging Demands , 2013, IEEE Transactions on Power Systems.

[3]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[4]  Michael Hyland,et al.  Dynamic autonomous vehicle fleet operations: Optimization-based strategies to assign AVs to immediate traveler demand requests , 2018, Transportation Research Part C: Emerging Technologies.

[5]  Paolo Santi,et al.  Supporting Information for Quantifying the Benefits of Vehicle Pooling with Shareability Networks Data Set and Pre-processing , 2022 .

[6]  L. Burns Sustainable mobility: A vision of our transport future , 2013, Nature.

[7]  Kara M. Kockelman,et al.  Operations of a Shared, Autonomous Electric Vehicle Fleet: Implications of Vehicle & Charging Infrastructure Decisions , 2016 .

[8]  Warren B. Powell,et al.  An Approximate Dynamic Programming Algorithm for Monotone Value Functions , 2014, Oper. Res..

[9]  Yazhe Wang,et al.  Understanding the effects of taxi ride-sharing - A case study of Singapore , 2018, Comput. Environ. Urban Syst..

[10]  Asif Weaver Christopher S. Walsh Michael P. Faiz,et al.  Air Pollution from Motor Vehicles: Standards and Technologies for Controlling Emissions , 1997 .

[11]  Ming Yang,et al.  Dynamic fleet management for cybercars , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[12]  Ruud H. Teunter,et al.  Optimal pricing for ride-sourcing platforms , 2019, Eur. J. Oper. Res..

[13]  Warren B. Powell,et al.  An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application , 2009, Transp. Sci..

[14]  Nikolas Geroliminis,et al.  An optimization framework for the development of efficient one-way car-sharing systems , 2015, Eur. J. Oper. Res..

[15]  Warren B. Powell,et al.  A Representational Paradigm for Dynamic Resource Transformation Problems , 2001, Ann. Oper. Res..

[16]  Nidhi Kalra,et al.  Autonomous Vehicle Technology: A Guide for Policymakers , 2014 .

[17]  Patrick Jaillet,et al.  Online Vehicle Routing: The Edge of Optimization in Large-Scale Applications , 2019, Oper. Res..

[18]  Hubertus Bardt Deutsche Autoindustrie und autonomes Fahren , 2016 .

[19]  Todd Litman,et al.  Autonomous Vehicle Implementation Predictions: Implications for Transport Planning , 2015 .

[20]  Regina R. Clewlow,et al.  Disruptive Transportation: The Adoption, Utilization, and Impacts of Ride-Hailing in the United States , 2017 .

[21]  Gereon Meyer,et al.  European Roadmap Smart Systems for Automated Driving , 2015 .

[22]  Hani S. Mahmassani,et al.  Evaluating the impact of spatio-temporal demand forecast aggregation on the operational performance of shared autonomous mobility fleets , 2019, Transportation.

[23]  Warren B. Powell,et al.  Hierarchical Knowledge Gradient for Sequential Sampling , 2011, J. Mach. Learn. Res..

[24]  Warren B. Powell,et al.  C Xxxx Society for Industrial and Applied Mathematics Optimal Learning in Experimental Design Using the Knowledge Gradient Policy with Application to Characterizing Nanoemulsion Stability , 2022 .

[25]  Caroline J Rodier,et al.  Dynamic Ridesharing: Exploration of Potential for Reduction in Vehicle Miles Traveled , 2016 .

[26]  A. Emadi,et al.  Transportation 2.0 , 2011, IEEE Power and Energy Magazine.

[27]  A. Majumdar,et al.  Opportunities and challenges for a sustainable energy future , 2012, Nature.

[28]  Giovanni Pau,et al.  Internet of Vehicles: From intelligent grid to autonomous cars and vehicular fogs , 2016, Int. J. Distributed Sens. Networks.

[29]  Yi Guo,et al.  Two-Stage Economic Operation of Microgrid-Like Electric Vehicle Parking Deck , 2016, IEEE Transactions on Smart Grid.

[30]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[31]  Tong Chen,et al.  Management of a shared, autonomous, electric vehicle fleet : vehicle choice, charging infrastructure & pricing strategies , 2015 .

[32]  Stephen D. Boyles,et al.  A general framework for modeling shared autonomous vehicles with dynamic network-loading and dynamic ride-sharing application , 2017, Comput. Environ. Urban Syst..

[33]  Martin W. P. Savelsbergh,et al.  Optimization for dynamic ride-sharing: A review , 2012, Eur. J. Oper. Res..

[34]  Prateek Bansal Charging of Electric Vehicles: Technology and Policy Implications , 2015 .

[35]  Ward Whitt,et al.  Optimal battery purchasing and charging strategy at electric vehicle battery swap stations , 2019, Eur. J. Oper. Res..

[36]  Edieal J. Pinker Introduction to the OR Forum Article: "Tenure Analytics: Models for Predicting Research Impact" , 2015, Oper. Res..

[37]  Warren B. Powell,et al.  Monte Carlo Tree Search with Sampled Information Relaxation Dual Bounds , 2017, ArXiv.

[38]  Mario Gerla,et al.  Internet of vehicles: From intelligent grid to autonomous cars and vehicular clouds , 2014, 2014 IEEE World Forum on Internet of Things (WF-IoT).

[39]  Michael Himmelsbach,et al.  Autonomous Ground Vehicles—Concepts and a Path to the Future , 2012, Proceedings of the IEEE.

[40]  Mohan M. Trivedi,et al.  Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis , 2013, IEEE Transactions on Intelligent Transportation Systems.

[41]  J. Michael Harrison,et al.  Bayesian Dynamic Pricing Policies: Learning and Earning Under a Binary Prior Distribution , 2011, Manag. Sci..

[42]  Nidhi Kalra,et al.  Autonomous Vehicle Technology , 2014 .

[43]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[44]  Jakob Puchinger,et al.  A survey of models and algorithms for optimizing shared mobility , 2019, Transportation Research Part B: Methodological.

[45]  Sebastian Thrun,et al.  Towards fully autonomous driving: Systems and algorithms , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[46]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[47]  Peter Davidson,et al.  AUTONOMOUS VEHICLES - WHAT COULD THIS MEAN FOR THE FUTURE OF TRANSPORT? , 2015 .

[48]  Siddhartha Banerjee,et al.  The Value of State Dependent Control in Ridesharing Systems , 2018 .

[49]  Mohamed A. El-Sharkawi,et al.  Optimal Combined Bidding of Vehicle-to-Grid Ancillary Services , 2012, IEEE Transactions on Smart Grid.

[50]  M. G. Gelder,et al.  In Reply: BEHAVIOUR THERAPY , 1965 .

[51]  Wenquan Li,et al.  Research on Optimization of Vehicle Routing Problem for Ride-sharing Taxi , 2012 .

[52]  Warren B. Powell,et al.  An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, I: Single Period Travel Times , 2002, Transp. Sci..

[53]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[54]  Kara M. Kockelman,et al.  Analyzing the dynamic ride-sharing potential for shared autonomous vehicle fleets using cellphone data from Orlando, Florida , 2018, Comput. Environ. Urban Syst..

[55]  Kara M. Kockelman,et al.  The Travel and Environmental Implications of Shared Autonomous Vehicles, Using Agent-Based Model Scenarios , 2014 .

[56]  Panos Y. Papalambros,et al.  Autonomous Electric Vehicle Sharing System Design , 2015, DAC 2015.

[57]  Hajo A. Reijers,et al.  The Share-a-Ride Problem: People and parcels sharing taxis , 2014, Eur. J. Oper. Res..

[58]  Kara M. Kockelman,et al.  Fleet performance and cost evaluation of a shared autonomous electric vehicle (SAEV) fleet: A case study for Austin, Texas , 2019, Transportation Research Part A: Policy and Practice.

[59]  Sanjeev R. Kulkarni,et al.  Value Function Approximation using Hierarchical Aggregation for Multiattribute Resource Management , 2004 .

[60]  Warren B. Powell,et al.  The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response , 2016, ArXiv.

[61]  Gonzalo Seco-Granados,et al.  Fair Design of Plug-in Electric Vehicles Aggregator for V2G Regulation , 2012, IEEE Transactions on Vehicular Technology.

[62]  Keemin Sohn,et al.  A systematic decision criterion for the elimination of useless overpasses , 2008 .

[63]  Warrren B Powell,et al.  Value Function Approximation using Multiple Aggregation for Multiattribute Resource Management , 2008 .

[64]  Itf Urban Mobility System Upgrade: How shared self-driving cars could change city traffic , 2015 .

[65]  Lei Ying,et al.  Empty-Car Routing in Ridesharing Systems , 2016, Oper. Res..

[66]  Warren B. Powell,et al.  Dynamic-Programming Approximations for Stochastic Time-Staged Integer Multicommodity-Flow Problems , 2006, INFORMS J. Comput..

[67]  Takeo Kanade,et al.  Autonomous land vehicle project at CMU , 1986, CSC '86.

[68]  M. Sivak,et al.  Potential impact of self-driving vehicles on household vehicle demand and usage , 2015 .

[69]  Jean-Marie Tarascon,et al.  Li-O2 and Li-S batteries with high energy storage. , 2011, Nature materials.