Approximating Values of Generalized-Reachability Stochastic Games

Simple stochastic games are turn-based 2½-player games with a reachability objective. The basic question asks whether one player can ensure reaching a given target with at least a given probability. A natural extension is games with a conjunction of such conditions as objective. Despite a plethora of recent results on the analysis of systems with multiple objectives, the decidability of this basic problem remains open. In this paper, we present an algorithm approximating the Pareto frontier of the achievable values to a given precision. Moreover, it is an anytime algorithm, meaning it can be stopped at any time returning the current approximation and its error bound.

[1]  Geoffrey J. Gordon,et al.  Bounded real-time dynamic programming: RTDP with monotone upper bounds and performance guarantees , 2005, ICML.

[2]  Krishnendu Chatterjee,et al.  Verification of Markov Decision Processes Using Learning Algorithms , 2014, ATVA.

[3]  Krishnendu Chatterjee,et al.  Gist: A Solver for Probabilistic Games , 2010, CAV.

[4]  Ufuk Topcu,et al.  Strategy Synthesis for Stochastic Games with Multiple Long-Run Objectives , 2015, TACAS.

[5]  Hongyang Qu,et al.  Quantitative Multi-objective Verification for Probabilistic Systems , 2011, TACAS.

[6]  Krishnendu Chatterjee,et al.  Perfect-Information Stochastic Games with Generalized Mean-Payoff Objectives* , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[7]  Taolue Chen,et al.  PRISM-games: A Model Checker for Stochastic Multi-Player Games , 2013, TACAS.

[8]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[9]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[10]  Yaron Velner.  Robust Multidimensional Mean-Payoff Games are Undecidable , 2015, FoSSaCS.

[11]  Jan Křetínský.  Approximating Values of Generalized-Reachability Stochastic Games: Invited Presentation at the 11th International Symposium on Games, Automata, Logics, and Formal Verification , 2020 .

[12]  Krishnendu Chatterjee,et al.  Multi-objective Discounted Reward Verification in Graphs and MDPs , 2013, LPAR.

[13]  Krishnendu Chatterjee,et al.  Markov Decision Processes with Multiple Long-Run Average Objectives , 2007, FSTTCS.

[14]  Kousha Etessami,et al.  Multi-Objective Model Checking of Markov Decision Processes , 2007, Log. Methods Comput. Sci..

[15]  Taolue Chen,et al.  Synthesis for Multi-objective Stochastic Games: An Application to Autonomous Urban Driving , 2013, QEST.

[16]  Marta Z. Kwiatkowska,et al.  PRISM-Games 2.0: A Tool for Multi-objective Strategy Synthesis for Stochastic Games , 2016, TACAS.

[17]  Marta Z. Kwiatkowska,et al.  Compositional strategy synthesis for stochastic games with multiple objectives , 2017, Inf. Comput..

[18]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2017, Formal Methods Syst. Des..

[19]  Krishnendu Chatterjee,et al.  Value Iteration , 2008, 25 Years of Model Checking.

[20]  Christel Baier,et al.  Trade-off analysis meets probabilistic model checking , 2014, CSL-LICS.

[21]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[22]  Romain Brenguier,et al.  Decidability Results for Multi-objective Stochastic Games , 2016, ATVA.

[23]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2014, CAV.

[24]  Alois Knoll,et al.  GAVS+: An Open Platform for the Research of Algorithmic Game Solving , 2011, TACAS.

[25]  Christel Baier,et al.  Energy-Utility Quantiles , 2014, NASA Formal Methods.

[26]  Krishnendu Chatterjee,et al.  Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2011, 2011 IEEE 26th Annual Symposium on Logic in Computer Science.

[27]  R. Ho.  Algebraic Topology , 2022 .

[28]  Benjamin Monmege,et al.  Interval iteration algorithm for MDPs and IMDPs , 2017, Theor. Comput. Sci..

[29]  Mihalis Yannakakis,et al.  On the approximability of trade-offs and optimal access of Web sources , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[30]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[31]  Krishnendu Chatterjee,et al.  Value Iteration for Long-Run Average Reward in Markov Decision Processes , 2017, CAV.

[32]  Jean-François Raskin,et al.  Pareto Curves of Multidimensional Mean-Payoff Games , 2015, CAV.

[33]  Krishnendu Chatterjee,et al.  A reduction from parity games to simple stochastic games , 2011, GandALF.

[34]  Krishnendu Chatterjee,et al.  MultiGain: A Controller Synthesis Tool for MDPs with Multiple Mean-Payoff Objectives , 2015, TACAS.

[35]  Marta Z. Kwiatkowska,et al.  PRISM-games: verification and strategy synthesis for stochastic multi-player games with multiple objectives , 2017, International Journal on Software Tools for Technology Transfer.

[36]  Christel Baier,et al.  Principles of model checking , 2008 .

[37]  Marta Z. Kwiatkowska,et al.  Quantitative verification and strategy synthesis for stochastic games , 2016, European Journal of Control.

[38]  Markus Lohrey,et al.  Computing quantiles in Markov chains with multi-dimensional costs , 2017, 2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[39]  Krishnendu Chatterjee,et al.  Trading Performance for Stability in Markov Decision Processes , 2013, 2013 28th Annual ACM/IEEE Symposium on Logic in Computer Science.

[40]  Krishnendu Chatterjee,et al.  Unifying Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2015, 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science.

[41]  Marta Z. Kwiatkowska,et al.  Pareto Curves for Probabilistic Model Checking , 2012, ATVA.

[42]  Jan Kretínský,et al.  Value Iteration for Simple Stochastic Games: Stopping Criterion and Learning Algorithm , 2018, CAV.

[43]  Jan Kretínský,et al.  Efficient Strategy Iteration for Mean Payoff in Markov Decision Processes , 2017, ATVA.

[44]  Christel Baier,et al.  Probabilistic Model Checking and Non-standard Multi-objective Reasoning , 2014, FASE.

[45]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[46]  D. Krass,et al.  Percentile performance criteria for limiting average Markov decision processes , 1995, IEEE Trans. Autom. Control..

[47]  Taolue Chen,et al.  On Stochastic Games with Multiple Objectives , 2013, MFCS.

[48]  Anne Condon,et al.  On Algorithms for Simple Stochastic Games , 1990, Advances In Computational Complexity Theory.