Countable-state-space Markov chains with two time scales and applications to queueing systems

Motivated by various applications in queueing systems, this work is devoted to continuous-time Markov chains with countable state spaces that involve both fast-time scale and slow-time scale with the aim of approximating the time-varying queueing systems by their quasistationary counterparts. Under smoothness conditions on the generators, asymptotic expansions of probability vectors and transition probability matrices are constructed. Uniform error bounds are obtained, and then sequences of occupation measures and their functionals are examined. Mean square error estimates of a sequence of occupation measures are obtained; a scaled sequence of functionals of occupation measures is shown to converge to a Gaussian process with zero mean. The representation of the variance of the limit process is also explicitly given. The results obtained are then applied to treat M t /M t /1 queues and Markov-modulated fluid buffer models.

[1]  Gang George Yin,et al.  Asymptotic Expansions of Singularly Perturbed Systems Involving Rapidly Fluctuating Markov Chains , 1996, SIAM J. Appl. Math..

[2]  François Baccelli,et al.  Elements Of Queueing Theory , 1994 .

[3]  R. Has’minskiĭ.  On Stochastic Processes Defined by Differential Equations with a Small Parameter , 1966 .

[4]  J. Harrison,et al.  Brownian motion and stochastic flow systems , 1986 .

[5]  A. Skorokhod,et al.  Studies in the theory of random processes , 1966 .

[6]  V. Lakshmikantham,et al.  Differential equations in abstract spaces , 1972 .

[7]  Marcel F. Neuts,et al.  Matrix-Geometric Solutions in Stochastic Models , 1981 .

[8]  W. Grassman.  Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory (Harold J. Kushner) , 1986 .

[9]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[10]  Ward Whitt,et al.  The Physics of the Mt/G/∞ Queue , 1993, Oper. Res..

[11]  W. A. Massey,et al.  Uniform acceleration expansions for Markov chains with time-varying rates , 1998 .

[12]  N. Bogolyubov,et al.  Asymptotic Methods in the Theory of Nonlinear Oscillations , 1961 .

[13]  T. Başar,et al.  H∞-Control of Markovian Jump Systems and Solutions to Associated Piecewise-Deterministic Differential Games , 1995 .

[14]  乔花玲,et al.  关于Semigroups of Linear Operators and Applications to Partial Differential Equations的两个注解 , 2003 .

[15]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[16]  Charles Knessl,et al.  Asymptotic analysis of a state dependent M/G/1 queueing system , 1985 .

[17]  Mogens Bladt,et al.  A sample path approach to mean busy periods for Markov-modulated queues and fluids , 1994, Advances in Applied Probability.

[18]  R. Z. Khasminskii,et al.  Singularly Perturbed Switching Diffusions: Rapid Switchings and Fast Diffusions , 1999 .

[19]  Qing Zhang,et al.  Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states , 2000 .

[20]  Andrew L. Rukhin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 2001, Technometrics.

[21]  A. M. Ilʹin,et al.  Matching of Asymptotic Expansions of Solutions of Boundary Value Problems , 1992 .

[22]  Thomas L. Saaty,et al.  Elements of queueing theory , 2003 .

[23]  J. R. E. O’Malley.  Singular perturbation methods for ordinary differential equations , 1991 .

[24]  William A. Massey,et al.  Asymptotic Analysis of the Time Dependent M/M/1 Queue , 1985, Math. Oper. Res..