The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response

Utilities face the challenge of responding to power outages due to storms and ice damage, but most power grids are not equipped with sensors to pinpoint the precise location of the faults causing the outage. Instead, utilities have to depend primarily on phone calls (trouble calls) from customers who have lost power to guide the dispatching of utility trucks. In this paper, we develop a policy that routes a utility truck to restore outages in the power grid as quickly as possible, using phone calls to create beliefs about outages, but also using utility trucks as a mechanism for collecting additional information. This means that routing decisions change not only the physical state of the truck (as it moves from one location to another) and the grid (as the truck performs repairs), but also our belief about the network, creating the first stochastic vehicle routing problem that explicitly models information collection and belief modeling. We address the problem of managing a single utility truck, which we start by formulating as a sequential stochastic optimization model which captures our belief about the state of the grid. We propose a stochastic lookahead policy, and use Monte Carlo tree search (MCTS) to produce a practical policy that is asymptotically optimal. Simulation results show that the developed policy restores the power grid much faster compared to standard industry heuristics.

[1]  Harilaos N. Psaraftis,et al.  A Dynamic Programming Solution to the Single Vehicle Many-to-Many Immediate Request Dial-a-Ride Problem , 1980 .

[2]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[3]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[4]  Warren B. Powell,et al.  Approximate dynamic programming in transportation and logistics: a unified framework , 2012, EURO J. Transp. Logist..

[5]  Warren B. Powell,et al.  A Probability Model for Grid Faults Using Incomplete Information , 2017, IEEE Transactions on Smart Grid.

[6]  Sean David Whipple Predictive storm damage modeling and optimizing crew response to improve storm response operations , 2014 .

[7]  Teodor Gabriel Crainic,et al.  Fleet management and logistics , 1998 .

[8]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[9]  Otthein Herzog,et al.  Monte-Carlo Tree Search for Logistics , 2016 .

[10]  Alan Fern,et al.  Lower Bounding Klondike Solitaire with Monte-Carlo Planning , 2009, ICAPS.

[11]  Guillaume Chaslot,et al.  A Comparison of Monte-Carlo Methods for Phantom Go , 2007 .

[12]  David Auger,et al.  Continuous Upper Confidence Trees with Polynomial Exploration - Consistency , 2013, ECML/PKDD.

[13]  Gilbert Laporte,et al.  Dynamic pickup and delivery problems , 2010, Eur. J. Oper. Res..

[14]  Michel Gendreau,et al.  A Hybrid Monte Carlo Local Branching Algorithm for the Single Vehicle Routing Problem with Stochastic Demands , 2010, Transp. Sci..

[15]  Richard J. Campbell,et al.  Weather-Related Power Outages and Electric System Resiliency , 2012 .

[16]  Jacek Mandziuk,et al.  UCT-Based Approach to Capacitated Vehicle Routing Problem , 2015, ICAISC.

[17]  Russell Bent,et al.  Scenario-Based Planning for Partially Dynamic Vehicle Routing with Stochastic Customers , 2004, Oper. Res..

[18]  J.T. Lin,et al.  An Expert System for Locating Distribution System Faults , 1991, IEEE Power Engineering Review.

[19]  Tristan Cazenave,et al.  A Phantom-Go Program , 2006, ACG.

[20]  H N Psaraftis,et al.  DYNAMIC VEHICLE ROUTING PROBLEMS. VEHICLE ROUTING: METHODS AND STUDIES. STUDIES IN MANAGEMENT SCIENCE AND SYSTEMS - VOLUME 16 , 1988 .

[21]  Michel Gendreau,et al.  A review of dynamic vehicle routing problems , 2013, Eur. J. Oper. Res..

[22]  Warren B. Powell,et al.  Clearing the Jungle of Stochastic Optimization , 2014 .

[23]  N. N. Schulz,et al.  Knowldege-Based System for Distribution System Outage Locating Using Comprehensive Information , 2002, IEEE Power Engineering Review.

[24]  W. G. Scott,et al.  Automating the restoration of distribution services in major emergencies , 1990 .

[25]  Rémi Munos,et al.  From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning , 2014, Found. Trends Mach. Learn..

[26]  G. C. Lampley Fault detection and location on electrical distribution system , 2002, 2002 Rural Electric Power Conference. Papers Presented at the 46th Annual Conference (Cat. No. 02CH37360).

[27]  Marius M. Solomon,et al.  Partially dynamic vehicle routing—models and algorithms , 2002, J. Oper. Res. Soc..

[28]  Gilbert Laporte,et al.  An Integer L-Shaped Algorithm for the Capacitated Vehicle Routing Problem with Stochastic Demands , 2002, Oper. Res..

[29]  Nataliya Sokolovska,et al.  Continuous Upper Confidence Trees , 2011, LION.

[30]  V. Rich Out of sight out of mind? , 1975, Nature.

[31]  Warren B. Powell,et al.  Stochastic and dynamic networks and routing , 1995 .

[32]  Y.-C. Lin,et al.  An artificial neural network based trouble call analysis , 1994 .

[33]  Dimitri P. Bertsekas,et al.  Rollout Algorithms for Stochastic Scheduling Problems , 1999, J. Heuristics.

[34]  Barrett W. Thomas,et al.  Rollout Policies for Dynamic Solutions to the Multivehicle Routing Problem with Stochastic Demand and Duration Limits , 2013, Oper. Res..

[35]  Christos Douligeris,et al.  An integrated framework for managing emergency-response logistics: the case of the electric utility companies , 1998 .

[36]  Pieter Spronck,et al.  Monte-Carlo Tree Search: A New Framework for Game AI , 2008, AIIDE.

[37]  Fushuan Wen,et al.  Tabu search based approach to trouble call analysis , 1998 .

[38]  H. Wang,et al.  OPRO: Precise emergency preparedness for electric utilities , 2016, IBM J. Res. Dev..

[39]  Michael C. Fu,et al.  An Adaptive Sampling Algorithm for Solving Markov Decision Processes , 2005, Oper. Res..

[40]  N. N. Schulz,et al.  An improved algorithm to aid in post-heat storm restoration , 1999 .

[41]  Michel Gendreau,et al.  DYNAMIC VEHICLE ROUTING AND DISPATCHING , 1998 .

[42]  Christos D. Tarantilis,et al.  Dynamic Vehicle Routing Problems , 2014, Vehicle Routing.

[43]  David P. Morton,et al.  Stochastic Vehicle Routing with Random Travel Times , 2003, Transp. Sci..

[44]  Bruce L. Golden,et al.  Stochastic vehicle routing: A comprehensive approach , 1983 .

[45]  J. D. de Wit,et al.  Out of sight, out of mind: cognitive states alter the focus of attention. , 2013, Experimental psychology.

[46]  Rémi Munos,et al.  Bandit Algorithms for Tree Search , 2007, UAI.