Stochastic Games with Disjunctions of Multiple Objectives (Technical Report)

Stochastic games combine controllable and adversarial non-determinism with stochastic behavior and are a common tool in control, verification and synthesis of reactive systems facing uncertainty. In this paper, we study turn-based stochastic two-player games on graphs where the winning condition is to satisfy at least one reachability or safety objective from a given set of alternatives with at least some desired probability. These objectives are also known as disjunctive queries (DQs). We present a fine-grained overview of strategy and computational complexity and provide new lower and upper bounds for several variants of the problem. These results extend the previous literature on DQs significantly. We also propose a novel value iteration-style algorithm for approximating the set of Pareto optimal thresholds for a given DQ.

[1]  Maximilian Weininger,et al.  Stochastic Games with Disjunctions of Multiple Objectives , 2021, Electronic Proceedings in Theoretical Computer Science.

[2]  Krishnendu Chatterjee,et al.  Multi-objective Discounted Reward Verification in Graphs and MDPs , 2013, LPAR.

[3]  Roberto Bagnara,et al.  The Parma Polyhedra Library: Toward a complete set of numerical abstractions for the analysis and verification of hardware and software systems , 2006, Sci. Comput. Program..

[4]  Mickael Randour,et al.  Simple Strategies in Multi-Objective MDPs , 2020, TACAS.

[5]  Krishnendu Chatterjee,et al.  A Survey of Stochastic Games with Limsup and Liminf Objectives , 2009, ICALP.

[6]  Krishnendu Chatterjee,et al.  Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2011, 2011 IEEE 26th Annual Symposium on Logic in Computer Science.

[7]  Krishnendu Chatterjee,et al.  Perfect-Information Stochastic Games with Generalized Mean-Payoff Objectives* , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[8]  Christel Baier,et al.  Probabilistic Model Checking and Non-standard Multi-objective Reasoning , 2014, FASE.

[9]  Taolue Chen,et al.  On Stochastic Games with Multiple Objectives , 2013, MFCS.

[10]  Gabriel Santos,et al.  Multi-player Equilibria Verification for Concurrent Stochastic Games , 2020, QEST.

[11]  D. Krass,et al.  Percentile performance criteria for limiting average Markov decision processes , 1995, IEEE Trans. Autom. Control..

[12]  Krishnendu Chatterjee,et al.  Value Iteration , 2008, 25 Years of Model Checking.

[13]  Hongyang Qu,et al.  Quantitative Multi-objective Verification for Probabilistic Systems , 2011, TACAS.

[14]  Krishnendu Chatterjee,et al.  A survey of partial-observation stochastic parity games , 2012, Formal Methods in System Design.

[15]  Krishnendu Chatterjee,et al.  Stochastic Games with Lexicographic Reachability-Safety Objectives , 2020, CAV.

[16]  Krishnendu Chatterjee,et al.  A survey of stochastic ω-regular games , 2012, J. Comput. Syst. Sci..

[17]  Thomas A. Henzinger,et al.  A Survey of Bidding Games on Graphs (Invited Paper) , 2020, CONCUR.

[18]  Marta Z. Kwiatkowska,et al.  Compositional Controller Synthesis for Stochastic Games , 2014, CONCUR.

[19]  Taolue Chen,et al.  Synthesis for Multi-objective Stochastic Games: An Application to Autonomous Urban Driving , 2013, QEST.

[20]  Marta Z. Kwiatkowska,et al.  PRISM-games: verification and strategy synthesis for stochastic multi-player games with multiple objectives , 2017, International Journal on Software Tools for Technology Transfer.

[21]  Pranav Ashok,et al.  Approximating Values of Generalized-Reachability Stochastic Games , 2019, LICS.

[22]  Shibashis Guha,et al.  Mixing Probabilistic and non-Probabilistic Objectives in Markov Decision Processes , 2020, LICS.

[23]  Krishnendu Chatterjee,et al.  Unifying Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2015, 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science.

[24]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[25]  Gabriel Santos,et al.  PRISM-games 3.0: Stochastic Game Verification with Concurrency, Equilibria and Time , 2020, CAV.

[26]  Krishnendu Chatterjee,et al.  Strategy improvement for concurrent reachability and turn-based stochastic safety games☆☆☆ , 2013, J. Comput. Syst. Sci..

[27]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2014, CAV.

[28]  Jan Kretínský,et al.  Comparison of Algorithms for Simple Stochastic Games , 2020, GandALF.

[29]  Florian Horn,et al.  The surprizing complexity of reachability games , 2010, ArXiv.

[30]  Kousha Etessami,et al.  Efficient Qualitative Analysis of Classes of Recursive Markov Decision Processes and Simple Stochastic Games , 2006, STACS.

[31]  Kim G. Larsen,et al.  Online and Compositional Learning of Controllers with Application to Floor Heating , 2016, TACAS.

[32]  Ufuk Topcu,et al.  Controller synthesis for autonomous systems interacting with human operators , 2015, ICCPS.

[33]  Anne Condon,et al.  On Algorithms for Simple Stochastic Games , 1990, Advances In Computational Complexity Theory.

[34]  Jan Kretínský,et al.  PAC Statistical Model Checking for Markov Decision Processes and Stochastic Games , 2019, CAV.

[35]  Romain Brenguier,et al.  Decidability Results for Multi-objective Stochastic Games , 2016, ATVA.

[36]  Kousha Etessami,et al.  Multi-Objective Model Checking of Markov Decision Processes , 2007, Log. Methods Comput. Sci..