Traffic-Aware Adaptive Routing for Minimizing Fuel Consumption

Fuel consumption in the vehicles are dependent on two variables of the route-deterministic and stochastic variables. The deterministic variable comprises length, elevation, curvature of road and stop signs along the route. The stochastic variable comprises variability in the velocity due to traffic and traffic lights. In this paper, we formulate the problem of traffic aware adaptive routing as a Markov decision problem (MDP) with total cost criterion featuring a continuum of state and finite actions. We show that the value iteration algorithm associated with our model would converge. Since it is impractical to implement value iteration on continuous spaces, we use Q learning with function approximation to perform numerical simulations to learn routing policies.

[1]  Brian C. Dean,et al.  Algorithms for minimum‐cost paths in time‐dependent networks with waiting policies , 2004, Networks.

[2]  O. Hernández-Lerma.  Average optimality in dynamic programming on Borel spaces: unbounded costs and controls , 1991 .

[3]  John N. Tsitsiklis,et al.  Dynamic Shortest Paths in Acyclic Networks with Markovian Arc Costs , 1993, Oper. Res..

[4]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[5]  C. Daganzo.  THE CELL TRANSMISSION MODEL.. , 1994 .

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Dimitri P. Bertsekas,et al.  A Mixed Value and Policy Iteration Method for Stochastic Control with Universally Measurable Policies , 2013, Math. Oper. Res..

[10]  D. Blackwell.  Discounted Dynamic Programming , 1965 .

[11]  Song Gao,et al.  Best Routing Policy Problem in Stochastic Time-Dependent Networks , 2002 .

[12]  Chelsea C. White,et al.  A Heuristic Search Approach for a Nonstationary Stochastic Shortest Path Problem with Terminal Cost , 2002, Transp. Sci..

[13]  Raghvendra V. Cowlagi,et al.  Hierarchical Motion Planning With Dynamical Feasibility Guarantees for Mobile Robotic Vehicles , 2012, IEEE Transactions on Robotics.

[14]  Calin Belta,et al.  Controlling a network of signalized intersections from temporal logical specifications , 2015, 2015 American Control Conference (ACC).

[15]  Hossein Soroush,et al.  Path Preferences and Optimal Paths in Probabilistic Networks , 1985, Transp. Sci..

[16]  Pravin Varaiya,et al.  Max pressure control of a network of signalized intersections , 2013 .

[17]  Ismail Chabini,et al.  A New Algorithm for Shortest Paths in Discrete Dynamic Networks , 1997 .

[18]  Randolph W. Hall,et al.  The Fastest Path through a Network with Random Time-Dependent Travel Times , 1986, Transp. Sci..

[19]  Robert L. Smith,et al.  Fastest Paths in Time-dependent Networks for Intelligent Vehicle-Highway Systems Application , 1993, J. Intell. Transp. Syst..

[20]  Ariel Orda,et al.  Shortest-path and minimum-delay algorithms in networks with time-dependent edge-length , 1990, JACM.

[21]  Liping Fu,et al.  An adaptive routing algorithm for in-vehicle route guidance system with real-time information , 2001 .

[22]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[23]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Matthew Brand,et al.  Stochastic Shortest Paths Via Quasi-convex Maximization , 2006, ESA.

[25]  Calin Belta,et al.  Formal Methods for Control of Traffic Flow: Automated Control Synthesis from Finite-State Transition Models , 2017, IEEE Control Systems.

[26]  Chelsea C. White,et al.  Optimal vehicle routing with real-time traffic information , 2005, IEEE Transactions on Intelligent Transportation Systems.

[27]  S. Travis Waller,et al.  On the online shortest path problem with limited arc cost dependencies , 2002, Networks.

[28]  A. Shwartz,et al.  Handbook of Markov decision processes : methods and applications , 2002 .

[29]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .