Sliding Window Abstraction for Infinite Markov Chains

We present an on-the-fly abstraction technique for infinite-state continuous -time Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic modeling of biological systems. We approximate the transient probability distributions at various time instances by solving a sequence of dynamically constructed abstract models, each depending on the previous one. Each abstract model is a finite Markov chain that represents the behavior of the original, infinite chain during a specific time interval. Our approach provides complete information about probability distributions, not just about individual parameters like the mean. The error of each abstraction can be computed, and the precision of the abstraction refined when desired. We implemented the algorithm and demonstrate its usefulness and efficiency on several case studies from systems biology.

[1]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[2]  Parosh Aziz Abdulla,et al.  Verifying infinite Markov chains with a finite attractor or the global coarseness property , 2005, 20th Annual IEEE Symposium on Logic in Computer Science (LICS' 05).

[3]  Aaron Bensimon,et al.  DNA replication origins fire stochastically in fission yeast. , 2005, Molecular biology of the cell.

[4]  Micha Yadin,et al.  Randomization Procedures in the Computation of Cumulative-Time Distributions over Discrete State Markov Processes , 1984, Oper. Res..

[5]  Antonín Kucera.  Methods for Quantitative Analysis of Probabilistic Pushdown Automata , 2005, INFINITY.

[6]  Rajeev Alur,et al.  A Temporal Logic of Nested Calls and Returns , 2004, TACAS.

[7]  Lijun Zhang,et al.  Time-bounded model checking of infinite-state continuous-time Markov chains , 2008, 2008 8th International Conference on Application of Concurrency to System Design.

[8]  Parosh Aziz Abdulla,et al.  Reasoning about Probabilistic Lossy Channel Systems , 2000, CONCUR.

[9]  Kousha Etessami,et al.  Algorithmic Verification of Recursive Probabilistic State Machines , 2005, TACAS.

[10]  Werner Sandmann,et al.  A Computational Stochastic Modeling Formalism for Biological Networks , 2008 .

[11]  T. T. Soong,et al.  Book Reviews : INTRODUCTION TO STOCHASTIC PROCESSES E. Cinlar Prentice-Hall, 1975 , 1979 .

[12]  C. Rao,et al.  Control, exploitation and tolerance of intracellular noise , 2002, Nature.

[13]  E. Hairer,et al.  Solving Ordinary Differential Equations I , 1987 .

[14]  L. Breuer.  Introduction to Stochastic Processes , 2022, Statistical Methods for Climate Scientists.

[15]  A. Arkin,et al.  Stochastic mechanisms in gene expression. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Luca de Alfaro,et al.  Magnifying-Lens Abstraction for Markov Decision Processes , 2007, CAV.

[17]  Parosh Aziz Abdulla,et al.  Verification of probabilistic systems with faulty communication , 2005, Inf. Comput..

[18]  Cleve B. Moler,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later , 1978, SIAM Rev..

[19]  Marta Z. Kwiatkowska,et al.  Game-based Abstraction for Markov Decision Processes , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[20]  Kousha Etessami,et al.  Verifying Probabilistic Procedural Programs , 2004, FSTTCS.

[21]  Thomas A. Henzinger,et al.  Approximation of event probabilities in noisy cellular processes , 2011, Theor. Comput. Sci..

[22]  Brian Munsky,et al.  Reduction and solution of the chemical master equation using time scale separation and finite state projection. , 2006, The Journal of chemical physics.

[23]  Ian Stark,et al.  The Continuous pi-Calculus: A Process Algebra for Biochemical Modelling , 2008, CMSB.

[24]  E. Hairer,et al.  Stiff and differential-algebraic problems , 1991 .

[25]  M. Thattai,et al.  Intrinsic noise in gene regulatory networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[27]  J. Goutsias.  Quasiequilibrium approximation of fast reaction kinetics in stochastic biochemical systems. , 2005, The Journal of chemical physics.

[28]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[29]  Kishor S. Trivedi,et al.  Transient Analysis of Acyclic Markov Chains , 1987, Perform. Evaluation.

[30]  K. Burrage,et al.  A Krylov-based finite state projection algorithm for solving the chemical master equation arising in the discrete modelling of biological systems , 2006 .

[31]  G. A. Baker.  Essentials of Padé approximants , 1975 .

[32]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[33]  Donald Gross,et al.  The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes , 1984, Oper. Res..

[34]  J. Esparza,et al.  Model checking probabilistic pushdown automata , 2004, LICS 2004.

[35]  D. Sherrington.  Stochastic Processes in Physics and Chemistry , 1983 .

[36]  F. Krogh,et al.  Solving Ordinary Differential Equations , 2019, Programming for Computations - Python.

[37]  Johann Christoph Strelen.  Approximate Disaggregation--Aggregation Solutions For General Queueing Networks , 1997 .

[38]  D. Milutinovic,et al.  Process noise: an explanation for the fluctuations in the immune response during acute viral infection. , 2007, Biophysical journal.

[39]  J. Rawlings,et al.  Approximate simulation of coupled fast and slow reactions for stochastic chemical kinetics , 2002 .

[40]  A. Dinner,et al.  Signatures of combinatorial regulation in intrinsic biological noise , 2008, Proceedings of the National Academy of Sciences.

[41]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[42]  Lijun Zhang,et al.  Time-Bounded Model Checking of Infinite-State Continuous-Time Markov Chains , 2009, Fundam. Informaticae.

[43]  Werner Sandmann,et al.  A Numerical Aggregation Algorithm for the Enzyme-Catalyzed Substrate Conversion , 2006, CMSB.

[44]  B. Philippe,et al.  Transient Solutions of Markov Processes by Krylov Subspaces , 1995 .

[45]  Alexander Moshe Rabinovich,et al.  Quantitative Analysis of Probabilistic Lossy Channel Systems , 2003, ICALP.

[46]  J. Hasty,et al.  Noise-based switches and amplifiers for gene expression. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[47]  E. Hairer,et al.  Solving ordinary differential equations I (2nd revised. ed.): nonstiff problems , 1993 .

[48]  Rawatee Maharaj-Sharma.  Online Lecture Notes , 2005 .

[49]  P. Swain,et al.  Intrinsic and extrinsic contributions to stochasticity in gene expression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[50]  Kousha Etessami,et al.  Optimizing Büchi Automata , 2000, CONCUR.

[51]  A. Arkin,et al.  Stochastic kinetic analysis of developmental pathway bifurcation in phage lambda-infected Escherichia coli cells. , 1998, Genetics.

[52]  Henrik Ejersbo Jensen,et al.  Reachability Analysis of Probabilistic Systems by Successive Refinements , 2001, PAPM-PROBMIV.

[53]  Joost-Pieter Katoen,et al.  Three-Valued Abstraction for Continuous-Time Markov Chains , 2007, CAV.

[54]  J. Paulsson.  Summing up the noise in gene networks , 2004, Nature.

[55]  Anne Remke,et al.  Model checking structured infinite Markov chains , 2008 .

[56]  M. Khammash,et al.  The finite state projection algorithm for the solution of the chemical master equation. , 2006, The Journal of chemical physics.