Occupation Measures of Singularly Perturbed Markov Chains with Absorbing States

This paper develops asymptotic properties of singularly perturbed Markov chains with inclusion of absorbing states. It focuses on both unscaled and scaled occupation measures. Under mild conditions, a mean-square estimate is obtained. By averaging the fast components, we obtain an aggregated process. Although the aggregated process itself may be non-Markovian, its weak limit is a Markov chain with much smaller state space. Moreover, a suitably scaled sequence consisting of a component of scaled occupation measures and a component of the aggregated process is shown to converge to a pair of processes with a switching diffusion component.

[1]  P. Kokotovic,et al.  A singular perturbation approach to modeling and control of Markov chains , 1981 .

[2]  A. Skorokhod Asymptotic Methods in the Theory of Stochastic Differential Equations , 2008 .

[3]  G. Yin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 1997 .

[4]  Qing Zhang,et al.  A central limit theorem for singularly perturbed nonstationary finite state Markov chains , 1996 .

[5]  Qing Zhang,et al.  Hierarchical Decision Making in Stochastic Manufacturing Systems , 1994 .

[6]  H. Kushner Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[7]  Qing Zhang,et al.  Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states , 2000 .

[8]  V. G. Gaitsgori,et al.  Theory of Suboptimal Decisions , 1988 .

[9]  T. Kurtz Approximation of Population Processes , 1987 .

[10]  A. A. Pervozvanskiĭ,et al.  Theory of Suboptimal Decisions: Decomposition and Aggregation , 1988 .

[11]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[12]  Qing Zhang,et al.  Singularly Perturbed Markov Chains , 2000 .

[13]  T. Başar,et al.  H∞-Control of Markovian Jump Systems and Solutions to Associated Piecewise-Deterministic Differential Games , 1995 .

[14]  R. Z. Khasminskii,et al.  Constructing asymptotic series for probability distributions of Markov chains with weak and strong interactions , 1997 .

[15]  Qing Zhang,et al.  Structural properties of Markov chains with weak and strong interactions , 1997 .

[16]  R. Z. Khasminskij On the principle of averaging the Itov's stochastic differential equations , 1968, Kybernetika.

[17]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..

[18]  R. Has’minskiĭ On Stochastic Processes Defined by Differential Equations with a Small Parameter , 1966 .

[19]  Harold J. Kushner,et al.  Approximation and Weak Convergence Methods for Random Processes , 1984 .

[20]  Gang George Yin,et al.  Asymptotic Expansions of Singularly Perturbed Systems Involving Rapidly Fluctuating Markov Chains , 1996, SIAM J. Appl. Math..

[21]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .