Arena-Independent Finite-Memory Determinacy in Stochastic Games

We study stochastic zero-sum games on graphs, which are prevalent tools to model decision-making in presence of an antagonistic opponent in a random environment. In this setting, an important question is the one of strategy complexity: what kinds of strategies are sufficient or required to play optimally (e.g., randomization or memory requirements)? Our contributions further the understanding of arena-independent finite-memory (AIFM) determinacy, i.e., the study of objectives for which memory is needed, but in a way that only depends on limited parameters of the game graphs. First, we show that objectives for which pure AIFM strategies suffice to play optimally also admit pure AIFM subgame perfect strategies. Second, we show that we can reduce the study of objectives for which pure AIFM strategies suffice in two-player stochastic games to the easier study of one-player stochastic games (i.e., Markov decision processes). Third, we characterize the sufficiency of AIFM strategies through two intuitive properties of objectives. This work extends a line of research started on deterministic games in [BLO+20] to stochastic ones. [BLO+20] Patricia Bouyer, Stephane Le Roux, Youssouf Oualhadj, Mickael Randour, and Pierre Vandenhove. Games Where You Can Play Optimally with Arena-Independent Finite Memory. CONCUR 2020.

[1]  Patricia Bouyer,et al.  Bounding Average-energy Games , 2016, FoSSaCS.

[2]  Wolfgang Thomas,et al.  Church's Problem and a Tour through Automata Theory , 2008, Pillars of Computer Science.

[3]  Patricia Bouyer,et al.  Games Where You Can Play Optimally with Finite Memory , 2020, ArXiv.

[4]  Véronique Bruyère,et al.  Energy mean-payoff games , 2019, CONCUR.

[5]  Krishnendu Chatterjee,et al.  The complexity of multi-mean-payoff and multi-energy games , 2012, Inf. Comput..

[6]  Krishnendu Chatterjee,et al.  Generalized Parity Games , 2007, FoSSaCS.

[7]  Hugo Gimbert,et al.  Pure Stationary Optimal Strategies in Markov Decision Processes , 2007, STACS.

[8]  Taolue Chen,et al.  On Stochastic Games with Multiple Objectives , 2013, MFCS.

[9]  Véronique Bruyère,et al.  Meet Your Expectations With Guarantees: Beyond Worst-Case Synthesis in Quantitative Games , 2013, STACS.

[10]  Mickael Randour,et al.  Variations on the Stochastic Shortest Path Problem , 2014, VMCAI.

[11]  Ashutosh Trivedi,et al.  Playing Stochastic Games Precisely , 2012, CONCUR.

[12]  Hugo Gimbert,et al.  Pure and Stationary Optimal Strategies in Perfect-Information Stochastic Games with Global Preferences , 2009, ArXiv.

[13]  Krishnendu Chatterjee,et al.  A survey of stochastic ω-regular games , 2012, J. Comput. Syst. Sci..

[14]  Krishnendu Chatterjee,et al.  Stochastic Games with Lexicographic Reachability-Safety Objectives , 2020, CAV.

[15]  T. Henzinger,et al.  Trading memory for randomness , 2004, First International Conference on the Quantitative Evaluation of Systems, 2004. QEST 2004. Proceedings..

[16]  Krishnendu Chatterjee,et al.  Strategy synthesis for multi-dimensional quantitative objectives , 2012, Acta Informatica.

[17]  Krishnendu Chatterjee,et al.  The complexity of stochastic Müller games , 2012, Inf. Comput..

[18]  Krishnendu Chatterjee,et al.  Energy Parity Games , 2010, ICALP.

[19]  Christel Baier,et al.  Principles of model checking , 2008 .

[20]  R. Durrett.  Probability: Theory and Examples , 1993 .

[21]  Mickael Randour,et al.  Automated synthesis of reliable and efficient systems through game theory: a case study , 2012, ArXiv.

[22]  Kim G. Larsen,et al.  Average-energy games , 2015, Acta Informatica.

[23]  Extending finite-memory determinacy to multi-player games , 2018, Inf. Comput..

[24]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[25]  Hugo Gimbert,et al.  Two-Player Perfect-Information Shift-Invariant Submixing Stochastic Games Are Half-Positional , 2014, ArXiv.

[26]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[27]  Krishnendu Chatterjee,et al.  Perfect-Information Stochastic Games with Generalized Mean-Payoff Objectives* , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[28]  Kim G. Larsen,et al.  Infinite Runs in Weighted Timed Automata with Energy Constraints , 2008, FORMATS.

[29]  Mickael Randour,et al.  Simple Strategies in Multi-Objective MDPs , 2020, TACAS.

[30]  Hugo Gimbert,et al.  Games Where You Can Play Optimally Without Any Memory , 2005, CONCUR.

[31]  Eryk Kopczynski,et al.  Half-Positional Determinacy of Infinite Games , 2006, ICALP.

[32]  Aniello Murano,et al.  Exploring the boundary of half-positionality , 2011, Annals of Mathematics and Artificial Intelligence.

[33]  Arno Pauly,et al.  Extending finite-memory determinacy by Boolean combination of winning conditions , 2018, FSTTCS.

[34]  Mickael Randour,et al.  Life is Random, Time is Not: Markov Decision Processes with Window Objectives , 2019, CONCUR.

[35]  H. Gimbert,et al.  Submixing and shift-invariant stochastic games , 2014, International Journal of Game Theory.

[36]  Krishnendu Chatterjee,et al.  Unifying Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2015, 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science.

[37]  Anne Condon,et al.  The Complexity of Stochastic Games , 1992, Inf. Comput..

[38]  Krishnendu Chatterjee,et al.  Quantitative stochastic parity games , 2004, SODA '04.

[39]  Benjamin Aminof,et al.  First-cycle games , 2014, Inf. Comput..

[40]  Reaching Your Goal Optimally by Playing at Random , 2020, CONCUR.

[41]  Sven Schewe,et al.  MDPs with energy-parity objectives , 2017, 2017 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[42]  Kousha Etessami,et al.  One-Counter Stochastic Games , 2010, FSTTCS.

[43]  Bruce Bueno de Mesquita,et al.  An Introduction to Game Theory , 2014 .

[44]  Andrzej Wlodzimierz Mostowski,et al.  Regular expressions for infinite trees and a standard form of automata , 1984, Symposium on Computation Theory.

[45]  Krishnendu Chatterjee,et al.  Randomness for Free , 2010, MFCS.

[46]  Mickael Randour,et al.  Threshold Constraints with Guarantees for Parity Objectives in Markov Decision Processes , 2017, ICALP.

[47]  Benjamin Aminof,et al.  First-cycle games , 2017, Inf. Comput..

[48]  Krishnendu Chatterjee,et al.  Combinations of Qualitative Winning for Stochastic Parity Games , 2018, CONCUR.

[49]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2017, Formal Methods Syst. Des..

[50]  Véronique Bruyère,et al.  Window Parity Games: An Alternative Approach Toward Parity Games with Time Bounds (Full Version) , 2016, GandALF.

[51]  Krishnendu Chatterjee,et al.  Two Views on Multiple Mean-Payoff Objectives in Markov Decision Processes , 2011, 2011 IEEE 26th Annual Symposium on Logic in Computer Science.

[52]  Krishnendu Chatterjee,et al.  Partial-Observation Stochastic Games: How to Win When Belief Fails , 2011, 2012 27th Annual IEEE Symposium on Logic in Computer Science.