Cost vs. Time in Stochastic Games and Markov Automata

Costs and rewards are important tools for analysing quantitative aspects of models like energy consumption and costs of maintenance and repair. Under the assumption of transient costs, this paper considers the computation of expected cost-bounded rewards and cost-bounded reachability for Markov automata and stochastic games. We give a transformation of this class of properties to expected time-bounded rewards and time-bounded reachability, which can be computed by available algorithms. We prove the correctness of the transformation and show its effectiveness on a number of case studies.

[1]  Hongfei Fu,et al.  Verifying probabilistic systems: new algorithms and complexity results , 2014 .

[2]  Nicolás Wolovick,et al.  A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes , 2006, FORMATS.

[3]  Luca Benini,et al.  Dynamic power management for portable systems , 2000, MobiCom '00.

[4]  Bernd Becker,et al.  Abstraction-Based Computation of Reward Measures for Markov Automata , 2015, VMCAI.

[5]  Lijun Zhang,et al.  Time-Bounded Reachability Probabilities in Continuous-Time Markov Decision Processes , 2010, 2010 Seventh International Conference on the Quantitative Evaluation of Systems.

[6]  Joost-Pieter Katoen,et al.  Analysis of Timed and Long-Run Objectives for Markov Automata , 2014, Log. Methods Comput. Sci..

[7]  Peter Buchholz,et al.  Numerical analysis of continuous time Markov decision processes over finite horizons , 2011, Comput. Oper. Res..

[8]  Holger Hermanns,et al.  Model Checking Algorithms for Markov Automata , 2012, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[9]  Christel Baier,et al.  Reachability in continuous-time Markov reward decision processes , 2008, Logic and Automata.

[10]  Robert B. Ash,et al.  Probability & Measure Theory , 1999 .

[11]  Christel Baier,et al.  Principles of Model Checking (Representation and Mind Series) , 2008 .

[12]  Joost-Pieter Katoen,et al.  Discrete-Time Rewards Model-Checked , 2003, FORMATS.

[13]  Holger Hermanns,et al.  Interactive Markov Chains , 2002, Lecture Notes in Computer Science.

[14]  Roberto Segala,et al.  A Compositional Trace-Based Semantics for Probabilistic Automata , 1995, CONCUR.

[15]  Massoud Pedram,et al.  Stochastic modeling of a power-managed system-construction andoptimization , 2001, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[16]  Joost-Pieter Katoen,et al.  Modelling, Reduction and Analysis of Markov Automata (extended version) , 2013, QEST.

[17]  Martin R. Neuhäußer,et al.  Model checking nondeterministic and randomly timed systems , 2010 .

[18]  A. Tarski.  A LATTICE-THEORETICAL FIXPOINT THEOREM AND ITS APPLICATIONS , 1955 .

[19]  Joost-Pieter Katoen,et al.  Efficient Modelling and Generation of Markov Automata , 2012, CONCUR.

[20]  Mariëlle Stoelinga,et al.  A Rigorous, Compositional, and Extensible Framework for Dynamic Fault Tree Analysis , 2010, IEEE Transactions on Dependable and Secure Computing.

[21]  Jan Kretínský,et al.  Continuous-Time Stochastic Games with Time-Bounded Reachability , 2009, FSTTCS.

[22]  Joost-Pieter Katoen,et al.  Model checking Markov reward models with impulse rewards , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[23]  Sylvain Sorin,et al.  Stochastic Games and Applications , 2003 .

[24]  Bernd Becker,et al.  Cost vs. time in stochastic games and Markov automata , 2015, Formal Aspects of Computing.

[25]  B. L. Miller.  Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon , 1968 .

[26]  Hongfei Fu,et al.  Maximal Cost-Bounded Reachability Probability on Continuous-Time Markov Decision Processes , 2013, FoSSaCS.

[27]  Mariëlle Stoelinga,et al.  Modelling and Analysis of Markov Reward Automata , 2014, ATVA.

[28]  Lijun Zhang,et al.  A Semantics for Every GSPN , 2013, Petri Nets.

[29]  Christel Baier,et al.  On the Logical Characterisation of Performability Properties , 2000, ICALP.

[30]  Christel Baier,et al.  Principles of model checking , 2008 .

[31]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[32]  Lijun Zhang,et al.  On Probabilistic Automata in Continuous Time , 2010, 2010 25th Annual IEEE Symposium on Logic in Computer Science.

[33]  Sven Johr,et al.  Model checking compositional Markov systems , 2008 .

[34]  Greg N. Frederickson,et al.  Sequencing Tasks with Exponential Service Times to Minimize the Expected Flow Time or Makespan , 1981, JACM.

[35]  Joost-Pieter Katoen,et al.  Quantitative Timed Analysis of Interactive Markov Chains , 2012, NASA Formal Methods.

[36]  Holger Hermanns,et al.  Discrete-time rewards model-checked (to appear) , 2003 .