Lumpability for Uncertain Continuous-Time Markov Chains

The assumption of perfect knowledge of rate parameters in continuous-time Markov chains (CTMCs) is undermined when confronted with reality, where they may be uncertain due to lack of information or because of measurement noise. In this paper we consider uncertain CTMCs, where rates are assumed to vary non-deterministically with time from bounded continuous intervals. This leads to a semantics which associates each state with the reachable set of its probability under all possible choices of the uncertain rates. We develop a notion of lumpability which identifies a partition of states where each block preserves the reachable set of the sum of its probabilities, essentially lifting the wellknown CTMC ordinary lumpability to the uncertain setting. We proceed with this analogy with two further contributions: a logical characterization of uncertain CTMC lumping in terms of continuous stochastic logic; and a polynomial time and space algorithm for the minimization of uncertain CTMCs by partition refinement, using the CTMC lumping algorithm as an inner step. As a case study, we show that the minimizations in a substantial number of CTMC models reported in the literature are robust with respect to uncertainties around their original, fixed, rate values.

[1]  Luca Cardelli,et al.  Symbolic computation of differential equivalences , 2016, POPL.

[2]  Robert K. Brayton,et al.  Model-checking continuous-time Markov chains , 2000, TOCL.

[3]  Joost-Pieter Katoen,et al.  On the use of model checking techniques for dependability evaluation , 2000, Proceedings 19th IEEE Symposium on Reliable Distributed Systems SRDS-2000.

[4]  Joost-Pieter Katoen,et al.  A Markov reward model checker , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[5]  Lijun Zhang,et al.  Bisimulations and Logical Characterizations on Continuous-Time Markov Decision Processes , 2014, VMCAI.

[6]  Sebastian Junges,et al.  PROPhESY: A PRObabilistic ParamEter SYnthesis Tool , 2015, CAV.

[7]  G. Franceschinis,et al.  Bounds for Quasi-Lumpable Markow Chains , 1994, Perform. Evaluation.

[8]  J.H.F. Linssen,et al.  continuous-time markov decision processes , 2016 .

[9]  Diego Latella,et al.  Model checking dependability attributes of wireless group communication , 2004, International Conference on Dependable Systems and Networks, 2004.

[10]  J. M. Watt Numerical Initial Value Problems in Ordinary Differential Equations , 1972 .

[11]  Lijun Zhang,et al.  Model Checking Algorithms for CTMDPs , 2011, CAV.

[12]  Joost-Pieter Katoen,et al.  Robust PCTL model checking , 2012, HSCC '12.

[13]  Mahesh Viswanathan,et al.  Automatic Reachability Analysis for Nonlinear Hybrid Models with C2E2 , 2016, CAV.

[14]  D. Vere-Jones Markov Chains , 1972, Nature.

[15]  Luca Bortolussi,et al.  Fluid Model Checking , 2012, CONCUR.

[16]  Martin Leucker,et al.  Don't Know in Probabilistic Systems , 2006, SPIN.

[17]  Joost-Pieter Katoen,et al.  Three-Valued Abstraction for Continuous-Time Markov Chains , 2007, CAV.

[18]  Lev V. Utkin,et al.  Interval-Valued Finite Markov Chains , 2002, Reliab. Comput..

[19]  J. Tyson,et al.  Model scenarios for evolution of the eukaryotic cell cycle. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[20]  Lijun Zhang,et al.  Probabilistic reachability for parametric Markov models , 2010, International Journal on Software Tools for Technology Transfer.

[21]  H. Hermanns,et al.  Syntax , Semantics , Equivalences , and Axioms for MTIPP y , 1994 .

[22]  Lubos Brim,et al.  Precise parameter synthesis for stochastic biochemical systems , 2014, Acta Informatica.

[23]  Hassane Alla,et al.  Discrete, continuous, and hybrid Petri Nets , 2004 .

[24]  Paola Lecca,et al.  Cell Cycle Control in Eukaryotes: A BioSpi model , 2007, Electron. Notes Theor. Comput. Sci..

[25]  Jane Hillston,et al.  A compositional approach to performance modelling , 1996 .

[26]  Andrea Vandin,et al.  UTOPIC: Under-Approximation Through Optimal Control , 2019, QEST.

[27]  Lijun Zhang,et al.  Probabilistic Reachability for Parametric Markov Models , 2009, SPIN.

[28]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[29]  Holger Hermanns,et al.  Exploiting Robust Optimization for Interval Probabilistic Bisimulation , 2016, QEST.

[30]  Kim G. Larsen,et al.  On the Metric-Based Approximate Minimization of Markov Chains , 2017, ICALP.

[31]  Andrea Bondavalli,et al.  QoS Analysis of Group Communication Protocols in Wireless Environment , 2002 .

[32]  Joost-Pieter Katoen,et al.  Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes , 2007, CONCUR.

[33]  Marta Z. Kwiatkowska,et al.  Symmetry Reduction for Probabilistic Model Checking , 2006, CAV.

[34]  P. Buchholz Exact and ordinary lumpability in finite Markov chains , 1994, Journal of Applied Probability.

[35]  Bernd Becker,et al.  Transient Reward Approximation for Continuous-Time Markov Chains , 2015, IEEE Transactions on Reliability.

[36]  William J. Stewart,et al.  Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling , 2009 .

[37]  Stefan Milius,et al.  Efficient Coalgebraic Partition Refinement , 2017, CONCUR.

[38]  Daniel Liberzon,et al.  Calculus of Variations and Optimal Control Theory: A Concise Introduction , 2012 .

[39]  I. Kiss,et al.  Exact epidemic models on graphs using graph-automorphism driven lumping , 2010, Journal of mathematical biology.

[40]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.

[41]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[42]  Kim Guldstrand Larsen,et al.  Specification and refinement of probabilistic processes , 1991, [1991] Proceedings Sixth Annual IEEE Symposium on Logic in Computer Science.

[43]  B. Nordstrom FINITE MARKOV CHAINS , 2005 .

[44]  William H. Sanders,et al.  Optimal state-space lumping in Markov chains , 2003, Inf. Process. Lett..

[45]  Hassan S. Bakouch,et al.  Probability, Markov chains, queues, and simulation , 2011 .

[46]  Luca Cardelli,et al.  Guaranteed Error Bounds on Approximate Model Abstractions through Reachability Analysis , 2018, QEST.

[47]  Giuliana Franceschinis,et al.  Simple O(m logn) Time Markov Chain Lumping , 2010, TACAS.

[48]  Peter Buchholz,et al.  Numerical analysis of continuous time Markov decision processes over finite horizons , 2011, Comput. Oper. Res..

[49]  Luca Cardelli,et al.  Maximal aggregation of polynomial dynamical systems , 2017, Proceedings of the National Academy of Sciences.

[50]  Holger Hermanns,et al.  Optimal Continuous Time Markov Decisions , 2015, ATVA.

[51]  Rupak Majumdar,et al.  Approximate Time Bounded Reachability for CTMCs and CTMDPs: A Lyapunov Approach , 2018, QEST.

[52]  Khaled M. Elbassioni,et al.  Polynomial-Time Alternating Probabilistic Bisimulation for Interval MDPs , 2017, SETTA.

[53]  Kim G. Larsen,et al.  Constraint Markov Chains , 2011, Theor. Comput. Sci..

[54]  Luca Cardelli,et al.  ERODE: A Tool for the Evaluation and Reduction of Ordinary Differential Equations , 2017, TACAS.

[55]  Stefan Milius,et al.  Generic Partition Refinement and Weighted Tree Automata , 2018, FM.

[56]  Radha Jagadeesan,et al.  Metrics for Labeled Markov Systems , 1999, CONCUR.

[57]  D. Gillespie Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[58]  Mahesh Viswanathan,et al.  Model-Checking Markov Chains in the Presence of Uncertainties , 2006, TACAS.

[59]  Sebastian Junges,et al.  A Storm is Coming: A Modern Probabilistic Model Checker , 2017, CAV.

[60]  Luca Bortolussi,et al.  Model checking single agent behaviours by fluid approximation , 2015, Inf. Comput..

[61]  Sergiy Bogomolov,et al.  A Box-Based Distance between Regions for Guiding the Reachability Analysis of SpaceEx , 2012, CAV.

[62]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..