Online Energy-optimal Routing for Electric Vehicles with Combinatorial Multi-arm Semi-Bandit

We have seen a rapid growth in the adoption of electric vehicle in today’s commercial mobility service market, from carrying passengers to the delivery of goods. One cardinal issue concerning the operation of EVs is the optimal routing of EV fleets with limited battery capacity. In this study, we investigate the energy-optimal online routing problem for the fleet of EVs, which focuses on identifying real-time minimum electricity consumption paths (MECP) for multiple OD pairs with limited information. We develop a multi-OD combinatorial multi-arm semi-bandit model (MCMAB) that uses the fleet of EVs as sensors in the transportation network and promotes the utilization of common information shared by different OD pairs. We further enrich the model with the path elimination policy to obtain MECP of high confidence while significantly reducing the number of learning iterations and the number of explorations needed. We demonstrate the effectiveness of the MCMAB and the efficiency of the path elimination policy with comprehensive numerical experiments in Manhattan, NYC. The results show that the proposed online routing algorithms can achieve near-optimal MECPs efficiently, and the quality of the solutions is significantly better than using the shortest travel time paths as approximate MECPs.

[1]  Shipra Agrawal,et al.  Analysis of Thompson Sampling for the Multi-armed Bandit Problem , 2011, COLT.

[2]  Grant Covic,et al.  Design considerations for a contactless electric vehicle battery charger , 2005, IEEE Transactions on Industrial Electronics.

[3]  Peter Auer,et al.  UCB revisited: Improved regret bounds for the stochastic multi-armed bandit problem , 2010, Period. Math. Hung..

[4]  Peter H. Bauer,et al.  Optimal Stochastic Eco-Routing Solutions for Electric Vehicles , 2018, IEEE Transactions on Intelligent Transportation Systems.

[5]  Nicolò Cesa-Bianchi,et al.  Combinatorial Bandits , 2012, COLT.

[6]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[7]  Kai Zheng,et al.  Combinatorial Semi-Bandit in the Non-Stationary Environment , 2020, ArXiv.

[8]  J. Y. Yen,et al.  Finding the K Shortest Loopless Paths in a Network , 2007 .

[9]  Vigna Kumaran Ramachandaramurthy,et al.  A review on the state-of-the-art technologies of electric vehicle, its impacts and prospects , 2015 .

[10]  Csaba Szepesvári,et al.  Exploration-exploitation tradeoff using variance estimates in multi-armed bandits , 2009, Theor. Comput. Sci..

[11]  Wtt Wtt.  Tight Regret Bounds for Stochastic Combinatorial Semi-Bandits , 2015 .

[12]  Satish V. Ukkusuri,et al.  Optimal charging facility location and capacity for electric vehicles considering route choice and charging time equilibrium , 2020, Comput. Oper. Res..

[13]  John Langford,et al.  The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information , 2007, NIPS.

[14]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[15]  Ricardo Maia,et al.  Electric vehicle simulator for energy consumption studies in electric mobility systems , 2011, 2011 IEEE Forum on Integrated and Sustainable Transportation Systems.

[16]  Richard Combes,et al.  Stochastic Online Shortest Path Routing: The Value of Feedback , 2013, IEEE Transactions on Automatic Control.

[17]  Stanislav Sobolevsky,et al.  Stationary Spatial Charging Demand Distribution for Commercial Electric Vehicles in Urban Area , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[18]  Shie Mannor,et al.  Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems , 2006, J. Mach. Learn. Res..

[19]  Alois Knoll,et al.  A simulation-based heuristic for city-scale electric vehicle charging station placement , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[20]  Fang He,et al.  Optimal deployment of public charging stations for plug-in hybrid electric vehicles , 2013 .

[21]  Natarajan Gautam,et al.  Deriving Link Travel - Time Distributions via Stochastic Speed Processes , 2004, Transp. Sci..