Deciding the Value 1 Problem for Reachability in 1-Clock Decision Stochastic Timed Automata

We consider reachability objectives on an extension of stochastic timed automata (STA) with nondeterminism. Decision stochastic timed automata (DSTA) are Markov decision processes based on timed automata where delays are chosen randomly and choices between enabled edges are nondeterministic. Given a reachability objective, the value 1 problem asks whether a target can be reached with probability arbitrary close to 1. Simple examples show that the value can be 1 and yet no strategy ensures reaching the target with probability 1. In this paper, we prove that, the value 1 problem is decidable for single clock DSTA by non-trivial reduction to a simple almost-sure reachability problem on a finite Markov decision process. The e-optimal strategies are involved: the precise probability distributions, even if they do not change the winning nature of a state, impact the timings at which e-optimal strategies must change their decisions, and more surprisingly these timings cannot be chosen uniformly over the set of regions.

[1]  Christel Baier,et al.  Probabilistic and Topological Semantics for Timed Automata , 2007, FSTTCS.

[2]  Joost-Pieter Katoen,et al.  Reachability probabilities in Markovian Timed Automata , 2011, IEEE Conference on Decision and Control and European Control Conference.

[3]  Christel Baier,et al.  Almost-Sure Model Checking of Infinite Paths in One-Clock Timed Automata , 2008, 2008 23rd Annual IEEE Symposium on Logic in Computer Science.

[4]  Hugo Gimbert,et al.  Deciding the Value 1 Problem for Probabilistic Leaktight Automata , 2011, 2012 27th Annual IEEE Symposium on Logic in Computer Science.

[5]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[6]  Thomas A. Henzinger,et al.  Concurrent reachability games , 2007, Theor. Comput. Sci..

[7]  Christel Baier,et al.  Principles of model checking , 2008 .

[8]  Philippe Schnoebelen,et al.  Model Checking Timed Automata with One or Two Clocks , 2004, CONCUR.

[9]  Nicolás Wolovick,et al.  A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes , 2006, FORMATS.

[10]  Kim G. Larsen,et al.  Optimal infinite scheduling for multi-priced timed automata , 2008, Formal Methods Syst. Des..

[11]  Jan Kretínský,et al.  Stochastic Real-Time Games with Qualitative Timed Automata Objectives , 2010, CONCUR.

[12]  Krishnendu Chatterjee,et al.  Qualitative concurrent parity games , 2011, TOCL.

[13]  Nathalie Bertrand,et al.  Playing Optimally on Timed Automata with Random Delays , 2012, FORMATS.

[14]  Hugo Gimbert,et al.  Probabilistic Automata on Finite Words: Decidable and Undecidable Problems , 2010, ICALP.

[15]  Patricia Bouyer,et al.  Reachability in Stochastic Timed Games , 2009, ICALP.

[16]  Krishnendu Chatterjee,et al.  Decidable Problems for Probabilistic Automata on Infinite Words , 2011, 2012 27th Annual IEEE Symposium on Logic in Computer Science.