PH-graphs for analyzing shortest path problems with correlated traveling times

This paper presents a new approach to model weighted graphs with correlated weights at the edges. Such models are important to describe many real world problems like routing in computer networks or finding shortest paths in traffic models under realistic assumptions. Edge weights are modeled by phase type distributions (PHDs), a versatile class of distributions based on continuous time Markov chains (CTMCs). Correlations between edge weights are introduced by adding dependencies between the PHDs of adjacent edges using transfer matrices. The new model class, denoted as PH graphs (PHGs), allows one to formulate many shortest path problems as the computation of an optimal policy in a continuous time Markov decision process (CTMDP). The basic model class is defined, methods to parameterize the required PHDs and transfer matrices based on measured data are introduced and the formulation of basic shortest path problems as solutions of CTMDPs with the corresponding solution algorithms are also provided. Numerical examples for some typical stochastic shortest path problems demonstrate the usability of the new approach.

[1]  Yueyue Fan,et al.  Shortest paths in stochastic networks with correlated link costs , 2005 .

[2]  Carrick Detweiler,et al.  Expected Shortest Paths for Landmark-Based Robot Navigation , 2004, Int. J. Robotics Res..

[3]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[4]  J. Tsitsiklis,et al.  Stochastic shortest path problems with recourse , 1996 .

[5]  Michael Mitzenmacher,et al.  Improved results for route planning in stochastic transportation , 2000, SODA '01.

[6]  Blai Bonet.  Solving Stochastic Shortest-Path Problems with RTDP , 2002 .

[7]  Søren Asmussen,et al.  Matrix‐analytic Models and their Analysis , 2000 .

[8]  C. O'Cinneide.  Characterization of phase-type distributions , 1990 .

[9]  Kishor S. Trivedi,et al.  Numerical transient analysis of markov models , 1988, Comput. Oper. Res..

[10]  Miklós Telek,et al.  PhFit: A General Phase-Type Fitting Tool , 2002, Computer Performance Evaluation / TOOLS.

[11]  David R. Karger,et al.  Route Planning under Uncertainty: The Canadian Traveller Problem , 2008, AAAI.

[12]  Michael L. Littman,et al.  Exact Solutions to Time-Dependent MDPs , 2000, NIPS.

[13]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[14]  Gábor Horváth,et al.  A minimal representation of Markov arrival processes and a moments matching method , 2007, Perform. Evaluation.

[15]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[16]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[17]  Randolph W. Hall,et al.  The Fastest Path through a Network with Random Time-Dependent Travel Times , 1986, Transp. Sci..

[18]  Song Gao,et al.  Optimal routing policy problems in stochastic time-dependent networks , 2006 .

[19]  Peter Buchholz,et al.  A MAP fitting approach with independent approximation of the inter-arrival time distribution and the lag correlation , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[20]  Wolfram Burgard,et al.  Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva , 2000, Int. J. Robotics Res..

[21]  Giovanni Andreatta,et al.  Stochastic shortest paths with recourse , 1988, Networks.

[22]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[23]  B. L. Miller.  Finite state continuous time Markov decision processes with an infinite planning horizon , 1968 .

[24]  Peter Buchholz,et al.  Numerical analysis of continuous time Markov decision processes over finite horizons , 2011, Comput. Oper. Res..

[25]  John N. Tsitsiklis,et al.  Stochastic shortest path problems with recourse , 1996, Networks.

[26]  Mihalis Yannakakis,et al.  Shortest Paths Without a Map , 1989, Theor. Comput. Sci..

[27]  David R. Karger,et al.  Optimal Route Planning under Uncertainty , 2006, ICAPS.

[28]  A. Horváth,et al.  Matching Three Moments with Minimal Acyclic Phase Type Distributions , 2005 .

[29]  András Varga,et al.  An overview of the OMNeT++ simulation environment , 2008, SimuTools.

[30]  Ren Asmussen,et al.  Fitting Phase-type Distributions via the EM Algorithm , 1996 .

[31]  Peter Buchholz,et al.  A Heuristic Approach for Fitting MAPs to Moments and Joint Moments , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[32]  Rudolf Hornig,et al.  An overview of the OMNeT++ simulation environment , 2008, Simutools 2008.

[33]  Richard F. Serfozo,et al.  Technical Note - An Equivalence Between Continuous and Discrete Time Markov Decision Processes , 1979, Oper. Res..

[34]  Ramin Sadre,et al.  Fitting World Wide Web request traces with the EM-algorithim , 2001, SPIE ITCom.

[35]  John N. Tsitsiklis,et al.  An Analysis of Stochastic Shortest Path Problems , 1991, Math. Oper. Res..

[36]  Marcel F. Neuts,et al.  Matrix-geometric solutions in stochastic models - an algorithmic approach , 1982 .

[37]  Peter Buchholz,et al.  A Novel Approach for Phase-Type Fitting with the EM Algorithm , 2006, IEEE Transactions on Dependable and Secure Computing.

[38]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[39]  Peter Buchholz,et al.  Transformation of Acyclic Phase Type Distributions for Correlation Fitting , 2013, ASMTA.

[40]  M. Neuts.  A Versatile Markovian Point Process , 1979 .

[41]  James J. Solberg,et al.  The Stochastic Shortest Route Problem , 1980, Oper. Res..

[42]  Axel Thümmler,et al.  Efficient phase-type fitting with aggregated traffic traces , 2007, Perform. Evaluation.

[43]  Dirk Sudholt,et al.  A Simple Ant Colony Optimizer for Stochastic Shortest Path Problems , 2012, Algorithmica.

[44]  Reid G. Simmons,et al.  Probabilistic Robot Navigation in Partially Observable Environments , 1995, IJCAI.

[45]  Peter Buchholz,et al.  Markov Modeling of Availability and Unavailability Data , 2014, 2014 Tenth European Dependable Computing Conference.

[46]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[47]  Manuel D. Rossetti.  Simulation Modeling and Arena , 2015 .

[48]  Miklós Telek,et al.  Matching More Than Three Moments with Acyclic Phase Type Distributions , 2007 .

[49]  B. L. Miller.  Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon , 1968 .

[50]  Michael P. Wellman,et al.  Path Planning under Time-Dependent Uncertainty , 1995, UAI.

[51]  Wolfgang Fischer,et al.  The Markov-Modulated Poisson Process (MMPP) Cookbook , 1993, Perform. Evaluation.

[52]  Miklós Telek,et al.  PhFit: a general phase-type fitting tool , 2002, Proceedings International Conference on Dependable Systems and Networks.

[53]  Peter Buchholz,et al.  ProFiDo - The Processes Fitting Toolkit Dortmund , 2010, 2010 Seventh International Conference on the Quantitative Evaluation of Systems.

[54]  C. O'Cinneide.  Phase-type distributions: open problems and a few properties , 1999 .

[55]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[56]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[57]  Philipp Reinecke,et al.  Phase-Type Distributions , 2012, Resilience Assessment and Evaluation of Computing Systems.

[58]  R. Serfozo.  An Equivalence between Continuous and Discrete Time Markov Decision Processes. , 1976 .