Decomposition and aggregation of large-dimensional Markov chains in discrete time

Motivated by a wide range of applications arising from stochastic networks (such as communication networks and/or manufacturing systems), this work focuses on a class of large-scale Markov chains in discrete time. In accordance with the rates of change of different states, we formulate the problem as a singularly perturbed Markov chain by introducing a small parameter /spl epsiv/>0. Under simple conditions, we show that aggregated process converges weakly to a Markov chain. In addition, we examine scaled and unscaled occupation measures and obtain their asymptotic properties.

[1]  Qing Zhang,et al.  Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states , 2000 .

[2]  Harold J. Kushner,et al.  Approximation and Weak Convergence Methods for Random Processes , 1984 .

[3]  Gang George Yin,et al.  On nearly optimal controls of hybrid LQG problems , 1999, IEEE Trans. Autom. Control..

[4]  P. Kokotovic,et al.  A singular perturbation approach to modeling and control of Markov chains , 1981 .

[5]  Gang George Yin,et al.  Singularly Perturbed Discrete-Time Markov Chains , 2000, SIAM J. Appl. Math..

[6]  R. H. Liu.  Nearly optimal control of singularly perturbed Markov decision processes in discrete time , 2001 .

[7]  John N. Tsitsiklis,et al.  Statistical Multiplexing of Multiple Time-Scale Markov Streams , 1995, IEEE J. Sel. Areas Commun..

[8]  G. Blankenship.  Singularly perturbed difference equations in optimal control problems , 1981 .

[9]  D. Naidu.  Singular Perturbation Methodology in Control Systems , 1988 .

[10]  R. Z. Khasminskii,et al.  Constructing asymptotic series for probability distributions of Markov chains with weak and strong interactions , 1997 .

[11]  Marius Iosifescu,et al.  Finite Markov Processes and Their Applications , 1981 .

[12]  Gang George Yin,et al.  Asymptotic Expansions of Singularly Perturbed Systems Involving Rapidly Fluctuating Markov Chains , 1996, SIAM J. Appl. Math..

[13]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[14]  G. Yin,et al.  Asymptotic Properties and Associated Control Problems of Discrete-Time Singularly Perturbed Markov Chains , 2002 .

[15]  J. Filar,et al.  Algorithms for singularly perturbed limiting average Markov control problems , 1992 .

[16]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[17]  Qing Zhang,et al.  Singularly Perturbed Markov Chains , 2000 .