Bounds on the deviation of discrete-time Markov chains from their mean-field model

We consider a generic mean-field scenario, in which a sequence of population models, described by discrete-time Markov chains (DTMCs), converges to a deterministic limit in discrete time. Under the assumption that the limit has a globally attracting equilibrium, the steady states of the sequence of DTMC models converge to the point-mass distribution concentrated on this equilibrium. In this paper we provide explicit bounds in probability for the convergence of such steady states, combining the stochastic bounds on the local error with control-theoretic tools used in the stability analysis of perturbed dynamical systems to bound the global accumulation of error. We also adapt this method to compute bounds on the transient dynamics. The approach is illustrated by a wireless sensor network example.

[1]  Jeremy T. Bradley,et al.  A fluid analysis framework for a Markovian process algebra , 2010, Theor. Comput. Sci..

[2]  J. Norris,et al.  Differential equation approximations for Markov chains , 2007, 0710.3269.

[3]  Adam Shwartz,et al.  Large Deviations For Performance Analysis , 2019 .

[4]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[5]  Stephen Gilmore,et al.  Analysing distributed Internet worm attacks using continuous state-space approximation of process algebra models , 2008, J. Comput. Syst. Sci..

[6]  Eduardo Sontag,et al.  Input-to-state stability for discrete-time nonlinear systems , 1999, at - Automatisierungstechnik.

[7]  Luca Bortolussi,et al.  Fluid Model Checking , 2012, CONCUR.

[8]  Zhong-Ping Jiang,et al.  A converse Lyapunov theorem for discrete-time systems with disturbances , 2002, Syst. Control. Lett..

[9]  D. V. Lindley,et al.  An Introduction to Probability Theory and Its Applications. Volume II , 1967, The Mathematical Gazette.

[10]  F. Chung,et al.  Complex Graphs and Networks , 2006 .

[11]  Oded Maler,et al.  Accurate hybridization of nonlinear systems , 2010, HSCC '10.

[12]  Boudewijn R. Haverkort,et al.  Mean-Field Analysis for the Evaluation of Gossip Protocols , 2009, QEST.

[13]  Donal O'Shea,et al.  Ideals, varieties, and algorithms - an introduction to computational algebraic geometry and commutative algebra (2. ed.) , 1997, Undergraduate texts in mathematics.

[14]  Frank E. Grubbs,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[15]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .

[16]  R. Hayden.  Convergence of ODE approximations and bounds on performance models in the steady-state , 2010 .

[17]  Alcherio Martinoli,et al.  Modeling Swarm Robotic Systems: a Case Study in Collaborative Distributed Manipulation , 2004, Int. J. Robotics Res..

[18]  Calin Belta,et al.  Controlling a Class of Nonlinear Systems on Rectangles , 2006, IEEE Transactions on Automatic Control.

[19]  M. Benaïm,et al.  A class of mean field interaction models for computer and communication systems , 2008, 2008 6th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks and Workshops.

[20]  Jean-Yves Le Boudec,et al.  A class of mean field interaction models for computer and communication systems , 2008, Perform. Evaluation.

[21]  Yuri Kifer,et al.  A Discrete-Time Version of the Wentzell-Friedlin Theory , 1990 .

[22]  Alberto Policriti,et al.  Dynamical Systems and Stochastic Programming: To Ordinary Differential Equations and Back , 2009, Trans. Comp. Sys. Biology.

[23]  Alexandre Proutière,et al.  A particle system in interaction with a rapidly varying environment: Mean field limits and applications , 2010, Networks Heterog. Media.

[24]  P. Olver.  Nonlinear Systems , 2013 .

[25]  Jeremy T. Bradley,et al.  Fluid computation of passage-time distributions in large Markov models , 2012, Theor. Comput. Sci..

[26]  Peter Key,et al.  Performance Analysis of Contention Based Medium Access Control Protocols , 2009, IEEE Trans. Inf. Theory.

[27]  Ward Whitt,et al.  An Introduction to Stochastic-Process Limits and their Application to Queues , 2002 .

[28]  Leonard Kleinrock,et al.  Theory, Volume 1, Queueing Systems , 1975 .

[29]  Allan Clark,et al.  Performance Specification and Evaluation with Unified Stochastic Probes and Fluid Analysis , 2013, IEEE Transactions on Software Engineering.

[30]  Holger Hermanns,et al.  Bounding the equilibrium distribution of Markov population models , 2010, Numer. Linear Algebra Appl..

[31]  W. Feller,et al.  An Introduction to Probability Theory and its Applications, Vol. II , 1967 .

[32]  Laurent Massoulié,et al.  Integrating streaming and file-transfer Internet traffic: fluid and diffusion approximations , 2007, Queueing Syst. Theory Appl..

[33]  Jean-Yves Le Boudec,et al.  A Generic Mean Field Convergence Result for Systems of Interacting Objects , 2007, Fourth International Conference on the Quantitative Evaluation of Systems (QEST 2007).

[34]  Oded Maler,et al.  Computing Reachable States for Nonlinear Biological Models , 2009, CMSB.

[35]  T. Kurtz.  Approximation of Population Processes , 1987 .