Bisimulations and Logical Characterizations on Continuous-Time Markov Decision Processes

In this paper we study strong and weak bisimulation equivalences for continuous-time Markov decision processes CTMDPs and the logical characterizations of these relations with respect to the continuous-time stochastic logic CSL. For strong bisimulation, it is well known that it is strictly finer than the CSL equivalence. In this paper we propose strong and weak bisimulations for CTMDPs and show that for a subclass of CTMDPs, strong and weak bisimulations are both sound and complete with respect to the equivalences induced by CSL and the sub-logic of CSL without next operator respectively. We then consider a standard extension of CSL, and show that it and its sub-logic without X can be fully characterized by strong and weak bisimulations respectively over arbitrary CTMDPs.

[1]  Sven Schewe,et al.  Finite optimal control for time-bounded reachability in CTMDPs and continuous-time Markov games , 2010, Acta Informatica.

[2]  R. J. vanGlabbeek.  The linear time - branching time spectrum , 1990 .

[3]  Bengt Jonsson,et al.  A logic for reasoning about time and reliability , 1990, Formal Aspects of Computing.

[4]  Edmund M. Clarke,et al.  Characterizing Finite Kripke Structures in Propositional Temporal Logic , 1988, Theor. Comput. Sci..

[5]  Radha Jagadeesan,et al.  Metrics for labelled Markov processes , 2004, Theor. Comput. Sci..

[6]  Nicolás Wolovick,et al.  A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes , 2006, FORMATS.

[7]  Lijun Zhang,et al.  Model Checking Interactive Markov Chains , 2010, TACAS.

[8]  Joost-Pieter Katoen,et al.  Weighted Lumpability on Markov Chains , 2011, Ershov Memorial Conference.

[9]  Carroll Morgan,et al.  Characterising Testing Preorders for Finite Probabilistic Processes , 2007, LICS.

[10]  Joseph Sifakis,et al.  Controller Synthesis for Timed Automata 1 , 1998 .

[11]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[12]  Ron Koymans,et al.  Specifying real-time properties with metric temporal logic , 1990, Real-Time Systems.

[13]  Joost-Pieter Katoen,et al.  Delayed Nondeterminism in Continuous-Time Markov Decision Processes , 2009, FoSSaCS.

[14]  A. Pnueli,et al.  CONTROLLER SYNTHESIS FOR TIMED AUTOMATA , 2006 .

[15]  Lijun Zhang,et al.  Flow Faster: Efficient Decision Algorithms for Probabilistic Simulations , 2007, Log. Methods Comput. Sci..

[16]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[17]  Lijun Zhang,et al.  The Branching Time Spectrum for Continuous-time MDPs , 2012, ArXiv.

[18]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[19]  Rob J. van Glabbeek,et al.  The Linear Time - Branching Time Spectrum I , 2001, Handbook of Process Algebra.

[20]  Joost-Pieter Katoen,et al.  The Ins and Outs of the Probabilistic Model Checker MRMC , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[21]  Joost-Pieter Katoen,et al.  On the use of model checking techniques for dependability evaluation , 2000, Proceedings 19th IEEE Symposium on Reliable Distributed Systems SRDS-2000.

[22]  Silvia Crafa,et al.  A Spectrum of Behavioral Relations over LTSs on Probability Distributions , 2011, CONCUR.

[23]  Adnan Aziz,et al.  It Usually Works: The Temporal Logic of Stochastic Systems , 1995, CAV.

[24]  Wang Yi,et al.  Probabilistic Extensions of Process Algebras , 2001, Handbook of Process Algebra.

[25]  Christel Baier,et al.  Comparative branching-time semantics for Markov chains , 2005, Inf. Comput..

[26]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[27]  Prakash Panangaden,et al.  Continuous stochastic logic characterizes bisimulation of continuous-time Markov processes , 2003, J. Log. Algebraic Methods Program..

[28]  Joël Ouaknine,et al.  The Cost of Punctuality , 2007, 22nd Annual IEEE Symposium on Logic in Computer Science (LICS 2007).

[29]  Christel Baier,et al.  Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes , 2005, Theor. Comput. Sci..

[30]  Thomas A. Henzinger,et al.  A really temporal logic , 1994, JACM.

[31]  Lijun Zhang,et al.  Probabilistic Logical Characterization , 2011, Inf. Comput..

[32]  Greg N. Frederickson,et al.  Sequencing Tasks with Exponential Service Times to Minimize the Expected Flow Time or Makespan , 1981, JACM.

[33]  Dejan Nickovic,et al.  From Mtl to Deterministic Timed Automata , 2010, FORMATS.

[34]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[35]  Nicolás Wolovick,et al.  Nondeterministic Labeled Markov Processes: Bisimulations and Logical Characterization , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[36]  Radha Jagadeesan,et al.  Weak bisimulation is sound and complete for pCTL* , 2002, Inf. Comput..

[37]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[38]  Lijun Zhang,et al.  Efficient Approximation of Optimal Control for Markov Games , 2010, ArXiv.

[39]  Bernd Becker,et al.  Compositional Performability Evaluation for STATEMATE , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[40]  Stuart Hannabuss.  Education: 2nd edition , 2000 .

[41]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[42]  Joël Ouaknine,et al.  On the decidability of metric temporal logic , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[43]  Roberto Segala,et al.  Comparative analysis of bisimulation relations on alternating and non-alternating probabilistic models , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[44]  W. Marsden.  I and J , 2012 .

[45]  Holger Hermanns,et al.  Uniformity by Construction in the Analysis of Nondeterministic Stochastic Systems , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[46]  Mathieu Tracol,et al.  Computing Distances between Probabilistic Automata , 2011, QAPL.

[47]  Rob J. van Glabbeek,et al.  The Linear Time - Branching Time Spectrum II , 1993, CONCUR.

[48]  Lijun Zhang,et al.  Bisimulations Meet PCTL Equivalences for Probabilistic Automata , 2011, CONCUR.

[49]  Peter Buchholz,et al.  Numerical analysis of continuous time Markov decision processes over finite horizons , 2011, Comput. Oper. Res..

[50]  Robert K. Brayton,et al.  Verifying Continuous Time Markov Chains , 1996, CAV.

[51]  Joël Ouaknine,et al.  Alternating Timed Automata over Bounded Time , 2010, 2010 25th Annual IEEE Symposium on Logic in Computer Science.

[52]  Nancy A. Lynch,et al.  Probabilistic Simulations for Probabilistic Processes , 1994, Nord. J. Comput..

[53]  Taolue Chen,et al.  Time-Bounded Verification of CTMCs against Real-Time Specifications , 2011, FORMATS.

[54]  Joost-Pieter Katoen,et al.  Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes , 2007, CONCUR.

[55]  Lijun Zhang,et al.  Model Checking Algorithms for CTMDPs , 2011, CAV.