Optimal Continuous Time Markov Decisions

In the context of Markov decision processes running in continuous time, one of the most intriguing challenges is the efficient approximation of finite horizon reachability objectives. A multitude of sophisticated model checking algorithms have been proposed for this. However, no proper benchmarking has been performed thus far.

[1]  Lijun Zhang,et al.  Model Checking Interactive Markov Chains , 2010, TACAS.

[2]  Peter Buchholz,et al.  Numerical analysis of continuous time Markov decision processes over finite horizons , 2011, Comput. Oper. Res..

[3]  GhemawatSanjay,et al.  The Google file system , 2003 .

[4]  Robert K. Brayton,et al.  Verifying Continuous Time Markov Chains , 1996, CAV.

[5]  Lijun Zhang,et al.  Efficient approximation of optimal control for continuous-time Markov games , 2016, Inf. Comput..

[6]  Lijun Zhang,et al.  Time-Bounded Reachability Probabilities in Continuous-Time Markov Decision Processes , 2010, 2010 Seventh International Conference on the Quantitative Evaluation of Systems.

[7]  Jan Kretínský,et al.  Continuous-Time Stochastic Games with Time-Bounded Reachability , 2013, FSTTCS.

[8]  Joost-Pieter Katoen,et al.  Modelling, Reduction and Analysis of Markov Automata (extended version) , 2013, QEST.

[9]  Joost-Pieter Katoen,et al.  Delayed Nondeterminism in Continuous-Time Markov Decision Processes , 2009, FoSSaCS.

[10]  Joost-Pieter Katoen,et al.  The How and Why of Interactive Markov Chains , 2009, FMCO.

[11]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[12]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[13]  Greg N. Frederickson,et al.  Sequencing Tasks with Exponential Service Times to Minimize the Expected Flow Time or Makespan , 1981, JACM.

[14]  Holger Hermanns,et al.  Improving time bounded reachability computations in interactive Markov chains , 2015, Sci. Comput. Program..

[15]  Joost-Pieter Katoen,et al.  On the use of model checking techniques for dependability evaluation , 2000, Proceedings 19th IEEE Symposium on Reliable Distributed Systems SRDS-2000.

[16]  Sven Schewe,et al.  Finite optimal control for time-bounded reachability in CTMDPs and continuous-time Markov games , 2010, Acta Informatica.

[17]  Claude Lefèvre,et al.  Optimal Control of a Birth and Death Epidemic Process , 1981, Oper. Res..

[18]  Dennis Guck,et al.  Quantitative Analysis of Markov Automata , 2012 .

[19]  Massoud Pedram,et al.  Stochastic modeling of a power-managed system-construction andoptimization , 2001, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[20]  Lijun Zhang,et al.  A Semantics for Every GSPN , 2013, Petri Nets.

[21]  Christel Baier,et al.  Efficient Computation of Time-Bounded Reachability Probabilities in Uniform Continuous-Time Markov Decision Processes , 2005, TACAS.

[22]  B. L. Miller.  Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon , 1968 .

[23]  Joost-Pieter Katoen,et al.  The Ins and Outs of the Probabilistic Model Checker MRMC , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[24]  Holger Hermanns,et al.  Model Checking Algorithms for Markov Automata , 2012, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[25]  Sven Schewe,et al.  Optimal time-abstract schedulers for CTMDPs and continuous-time Markov games , 2013, Theor. Comput. Sci..

[26]  Marco Ajmone Marsan,et al.  Modelling with Generalized Stochastic Petri Nets , 1995, PERV.

[27]  William H. Sanders,et al.  Stochastic Activity Networks: Structure, Behavior, and Application , 1985, PNPM.

[28]  Lijun Zhang,et al.  Model Checking Algorithms for CTMDPs , 2011, CAV.

[29]  Martin R. Neuhäußer,et al.  Model checking nondeterministic and randomly timed systems , 2010 .