Reachability in Augmented Interval Markov Chains

In this paper we propose augmented interval Markov chains (AIMCs): a generalisation of the familiar interval Markov chains (IMCs) where uncertain transition probabilities are in addition allowed to depend on one another. This new model preserves the flexibility afforded by IMCs for describing stochastic systems where the parameters are unclear, for example due to measurement error, but also allows us to specify transitions with probabilities known to be identical, thereby lending further expressivity. The focus of this paper is reachability in AIMCs. We study the qualitative, exact quantitative and approximate reachability problem, as well as natural subproblems thereof, and establish several upper and lower bounds for their complexity. We prove the exact reachability problem is at least as hard as the famous square-root sum problem, but, encouragingly, the approximate version lies in $\mathbf{NP}$ if the underlying graph is known, whilst the restriction of the exact problem to a constant number of uncertain edges is in $\mathbf{P}$. Finally, we show that uncertainty in the graph structure affects complexity by proving $\mathbf{NP}$-completeness for the qualitative subproblem, in contrast with an easily-obtained upper bound of $\mathbf{P}$ for the same subproblem with known graph structure.

[1]  Peter Bro Miltersen,et al.  2 The Task of a Numerical Analyst , 2022 .

[2]  Kim G. Larsen,et al.  Decision Problems for Interval Markov Chains , 2011, LATA.

[3]  Mahesh Viswanathan,et al.  Model-Checking Markov Chains in the Presence of Uncertainties , 2006, TACAS.

[4]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[5]  Krishnendu Chatterjee,et al.  Model-Checking omega-Regular Properties of Interval Markov Chains , 2008, FoSSaCS.

[6]  James Renegar,et al.  On the Computational Complexity and Geometry of the First-Order Theory of the Reals, Part I: Introduction. Preliminaries. The Geometry of Semi-Algebraic Sets. The Decision Problem for the Existential Theory of the Reals , 1992, J. Symb. Comput..

[7]  Jan J. M. M. Rutten,et al.  Mathematical techniques for analyzing concurrent and probabilistic systems , 2004, CRM monograph series.

[8]  Kim Guldstrand Larsen,et al.  Specification and refinement of probabilistic processes , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[9]  Mickael Randour,et al.  Percentile queries in multi-dimensional Markov decision processes , 2017, Formal Methods Syst. Des..

[10]  Marcus Schaefer,et al.  Fixed Points, Nash Equilibria, and the Existential Theory of the Reals , 2017, Theory of Computing Systems.

[11]  Laure Petrucci,et al.  Parameter Synthesis for Parametric Interval Markov Chains , 2016, VMCAI.

[12]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[13]  Lijun Zhang,et al.  Probabilistic reachability for parametric Markov models , 2010, International Journal on Software Tools for Technology Transfer.

[14]  Sebastian Junges,et al.  Synthesis in pMDPs: A Tale of 1001 Parameters , 2018, ATVA.

[15]  R. Bellman A Markovian Decision Process , 1957 .

[16]  Kousha Etessami,et al.  Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations , 2005, JACM.

[17]  Ronald L. Graham,et al.  Some NP-complete geometric problems , 1976, STOC '76.

[18]  A. Tarski A Decision Method for Elementary Algebra and Geometry , 2023 .

[19]  S. Smale,et al.  On a theory of computation and complexity over the real numbers; np-completeness , 1989 .

[20]  Mihalis Yannakakis,et al.  The complexity of probabilistic verification , 1995, JACM.

[21]  Sebastian Junges,et al.  Parameter Synthesis for Markov Models: Faster Than Ever , 2016, ATVA.

[22]  Prasoon Tiwari,et al.  A problem that is easier to solve on the unit-cost algebraic RAM , 1992, J. Complex..

[23]  Krishnendu Chatterjee,et al.  Robustness of Structurally Equivalent Concurrent Parity Games , 2011, FoSSaCS.

[24]  James Worrell,et al.  LTL Model Checking of Interval Markov Chains , 2013, TACAS.