Playing Optimally on Timed Automata with Random Delays

We marry continuous time Markov decision processes (CTMDPs) with stochastic timed automata into a model with joint expressive power. This extension is very natural, as the two original models already share exponentially distributed sojourn times in locations. It enriches CTMDPs with timing constraints, or symmetrically, stochastic timed automata with one conscious player. Our model maintains the existence of optimal control known for CTMDPs. This also holds for a richer model with two players, which extends continuous time Markov games. But we have to sacrifice the existence of simple schedulers: polyhedral regions are insufficient to obtain optimal control even in the single-player case.

[1]  Jan Kretínský,et al.  Stochastic Real-Time Games with Qualitative Timed Automata Objectives , 2010, CONCUR.

[2]  Pawel Sobocinski,et al.  Being Van Kampen is a universal property , 2011, Log. Methods Comput. Sci..

[3]  Patricia Bouyer,et al.  Reachability in Stochastic Timed Games , 2009, ICALP.

[4]  Sven Schewe,et al.  Finite optimal control for time-bounded reachability in CTMDPs and continuous-time Markov games , 2010, Acta Informatica.

[5]  Christel Baier,et al.  Efficient Computation of Time-Bounded Reachability Probabilities in Uniform Continuous-Time Markov Decision Processes , 2005, TACAS.

[6]  Joost-Pieter Katoen,et al.  Reachability probabilities in Markovian Timed Automata , 2011, IEEE Conference on Decision and Control and European Control Conference.

[7]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[8]  Jan Kretínský,et al.  Continuous-Time Stochastic Games with Time-Bounded Reachability , 2013, FSTTCS.

[9]  Lijun Zhang,et al.  Time-Bounded Reachability Probabilities in Continuous-Time Markov Decision Processes , 2010, 2010 Seventh International Conference on the Quantitative Evaluation of Systems.

[10]  Marta Z. Kwiatkowska,et al.  Automatic verification of real-time systems with discrete probability distributions , 1999, Theor. Comput. Sci..

[11]  Joost-Pieter Katoen,et al.  Quantitative Model Checking of Continuous-Time Markov Chains Against Timed Automata Specifications , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[12]  Jirí Srba,et al.  Comparing the Expressiveness of Timed Automata and Timed Extensions of Petri Nets , 2008, FORMATS.

[13]  Robin Milner,et al.  On Observing Nondeterminism and Concurrency , 1980, ICALP.

[14]  Christel Baier,et al.  Probabilistic and Topological Semantics for Timed Automata , 2007, FSTTCS.

[15]  Joost-Pieter Katoen,et al.  Observing Continuous-Time MDPs by 1-Clock Timed Automata , 2011, RP.

[16]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[17]  Joseph Sifakis,et al.  Controller Synthesis for Timed Automata 1 , 1998 .

[18]  Christel Baier,et al.  Almost-Sure Model Checking of Infinite Paths in One-Clock Timed Automata , 2008, 2008 23rd Annual IEEE Symposium on Logic in Computer Science.

[19]  Paul Gastin,et al.  CONCUR 2010 - Concurrency Theory, 21th International Conference, CONCUR 2010, Paris, France, August 31-September 3, 2010. Proceedings , 2010, CONCUR.

[20]  Lijun Zhang,et al.  Model Checking Interactive Markov Chains , 2010, TACAS.

[21]  Patricia Bouyer,et al.  Almost-Sure Model-Checking of Reactive Timed Automata , 2012, 2012 Ninth International Conference on Quantitative Evaluation of Systems.

[22]  Nicolás Wolovick,et al.  A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes , 2006, FORMATS.

[23]  A. Pnueli,et al.  CONTROLLER SYNTHESIS FOR TIMED AUTOMATA , 2006 .

[24]  Sanjiva Prasad,et al.  FSTTCS 2007: Foundations of Software Technology and Theoretical Computer Science, 27th International Conference, New Delhi, India, December 12-14, 2007, Proceedings , 2007, FSTTCS.