Optimization-based Lyapunov function construction for continuous-time Markov chains with affine transition rates

We address the problem of Lyapunov function construction for a class of continuous-time Markov chains with affine transition rates, typically encountered in stochastic chemical kinetics. Following an optimization approach, we take advantage of existing bounds from the Foster-Lyapunov stability theory to obtain functions that enable us to estimate the region of high stationary probability, as well as provide upper bounds on moments of the chain. Our method can be used to study the stationary behavior of a given chain without resorting to stochastic simulation, in a fast and efficient manner.

[1]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[2]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[3]  R. Tweedie.  Sufficient conditions for regularity, recurrence and ergodicity of Markov processes , 1975, Mathematical Proceedings of the Cambridge Philosophical Society.

[4]  R. Syski.  Passage Times for Markov Chains , 1992 .

[5]  S. Meyn,et al.  Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes , 1993, Advances in Applied Probability.

[6]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[7]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[8]  G. Fayolle,et al.  Topics in the Constructive Theory of Countable Markov Chains: Ideology of induced chains , 1995 .

[9]  G. Fayolle,et al.  Topics in the Constructive Theory of Countable Markov Chains , 1995 .

[10]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[11]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[12]  A. Papachristodoulou,et al.  On the construction of Lyapunov functions using the sum of squares decomposition , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[13]  Johan Efberg,et al.  YALMIP : A toolbox for modeling and optimization in MATLAB , 2004 .

[14]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[15]  M. Khammash,et al.  The finite state projection algorithm for the solution of the chemical master equation. , 2006, The Journal of chemical physics.

[16]  Holger Hermanns,et al.  Bounding the equilibrium distribution of Markov population models , 2010, Numer. Linear Algebra Appl..

[17]  Christian Mazza,et al.  Stochastic Dynamics for Systems Biology , 2014 .

[18]  Ankit Gupta,et al.  A Scalable Computational Framework for Establishing Long-Term Behavior of Stochastic Reaction Networks , 2013, PLoS Comput. Biol..