Variations on the Stochastic Shortest Path Problem

In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the distribution of the length of paths reaching a given target, rather than simply minimizing its expected value. The concepts and algorithms that we propose here are applications of more general results that have been obtained recently for Markov decision processes and that are described in a series of recent papers.

[1]  Andrew V. Goldberg,et al.  Shortest paths algorithms: Theory and experimental evaluation , 1994, SODA '94.

[2]  Krishnendu Chatterjee,et al.  Looking at mean-payoff and total-payoff through windows , 2015, Inf. Comput..

[3]  Véronique Bruyère,et al.  Meet Your Expectations With Guarantees: Beyond Worst-Case Synthesis in Quantitative Games , 2013, STACS.

[4]  Vladimir Gurvich,et al.  On Short Paths Interdiction Problems: Total and Node-Wise Limited Interdiction , 2008, Theory of Computing Systems.

[5]  Véronique Bruyère,et al.  Expectations or Guarantees? I Want It All! A crossroad between games and MDPs , 2014, SR.

[6]  Christel Baier,et al.  Principles of model checking , 2008 .

[7]  A. Ehrenfeucht,et al.  Positional strategies for mean payoff games , 1979 .

[8]  Yoshio Ohtsubo,et al.  Optimal threshold probability in undiscounted Markov decision processes with a target set , 2004, Appl. Math. Comput..

[9]  Benjamin Monmege,et al.  To Reach or not to Reach? Efficient Algorithms for Total-Payoff Games , 2014, CONCUR.

[10]  Krishnendu Chatterjee,et al.  What is decidable about partially observable Markov decision processes with ω-regular objectives , 2013, J. Comput. Syst. Sci..

[11]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2014, CAV.

[12]  Hongyang Qu,et al.  Quantitative Multi-objective Verification for Probabilistic Systems , 2011, TACAS.

[13]  Christel Baier,et al.  Computing Quantiles in Markov Reward Models , 2013, FoSSaCS.

[14]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[15]  Krishnendu Chatterjee,et al.  What is Decidable about Partially Observable Markov Decision Processes with omega-Regular Objectives , 2013, CSL.

[16]  Kousha Etessami,et al.  Multi-objective Model Checking of Markov Decision Processes , 2007, TACAS.

[17]  Jean-François Raskin,et al.  Quantitative Languages Defined by Functional Automata , 2011, CONCUR.

[18]  Moshe Y. Vardi.  Automatic verification of probabilistic concurrent finite state programs , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[19]  Yoshio Ohtsubo,et al.  Markov decision processes associated with two threshold probability criteria , 2013 .

[20]  Jean-François Raskin,et al.  Quantitative Languages Defined by Functional Automata , 2012, CONCUR.

[21]  John N. Tsitsiklis,et al.  An Analysis of Stochastic Shortest Path Problems , 1991, Math. Oper. Res..

[22]  Jean-François Raskin,et al.  Multiple-Environment Markov Decision Processes , 2014, FSTTCS.

[23]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[24]  Luca de Alfaro,et al.  Computing Minimum and Maximum Reachability Times in Probabilistic Systems , 1999, CONCUR.

[25]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[26]  Christoph Haase,et al.  The Odds of Staying on Budget , 2014, ICALP.