THE FUNDAMENTAL MATRIX OF SINGULARLY

We consider a singularly perturbed (finite state) Markov chain and provide a complete characterization of the fundamental matrix. In particular, we obtain a formula for the regular part simpler than a previous formula obtained by Schweitzer, and the singular part is obtained via a reduction process similar to Delebecque's reduction for the stationary distribution. In contrast to previous approaches, one works with aggregate Markov chains of much smaller dimension than the original chain, an essential feature for practical computation. An application to mean first-passage times is also presented.

[1]  J. Filar,et al.  Hamiltonian cycle problem and singularly perturbed Markov decision process , 1996 .

[2]  J. Lasserre.  A formula for singular perturbations of Markov chains , 1994, Journal of Applied Probability.

[3]  V. S. Koroli︠u︡k,et al.  Mathematical Foundations of the State Lumping of Large Systems , 1993 .

[4]  Moshe Haviv,et al.  On Series Expansions and Stochastic Matrices , 1993, SIAM J. Matrix Anal. Appl..

[5]  Refael Hassin,et al.  Mean Passage Times and Nearly Uncoupled Markov Chains , 1992, SIAM J. Discret. Math..

[6]  G. Stewart,et al.  The Laurent expansion of pencils that are singular at the origin , 1991 .

[7]  J. Filar,et al.  Algorithms for singularly perturbed limiting average Markov control problems , 1990, 29th IEEE Conference on Decision and Control.

[8]  G. Louchard,et al.  Geometric bounds on iterative approximations for nearly completely decomposable Markov chains , 1990, Journal of Applied Probability.

[9]  J. Filar,et al.  Singularly perturbed Markov control problem: Limiting average cost , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[10]  Alan S. Willsky,et al.  The reduction of perturbed Markov generators: an algorithm exposing the role of transient states , 1988, JACM.

[11]  V. G. Gaitsgori,et al.  Theory of Suboptimal Decisions , 1988 .

[12]  P J Schweitzer,et al.  Perturbation series expansions for nearly completely-decomposable Markov chains , 1986 .

[13]  H. Baumgärtel.  Analytic perturbation theory for matrices and operators , 1985 .

[14]  F. Delebecque.  A Reduction Process for Perturbed Markov Chains , 1983 .

[15]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[16]  P. Howlett,et al.  Input retrieval in finite dimensional linear systems , 1982, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[17]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..

[18]  Marius Iosifescu,et al.  Finite Markov Processes and Their Applications , 1981 .

[19]  G. Louchard,et al.  Return times in nearly-completely decomposable stochastic processes , 1978, Journal of Applied Probability.

[20]  Approximation of eigencharacteristics in nearly-completely decomposable stochastic systems , 1976 .

[21]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[22]  V. G. Gaitsgori,et al.  Aggregation of states in a Markov chain with weak interaction , 1975 .

[23]  A. A. Pervozvanskiĭ,et al.  Stationary-state evaluation for a complex system with slowly varying couplings , 1974 .

[24]  C. E. Langenhop,et al.  The Laurent expansion for a nearly singular matrix , 1971 .

[25]  J. Massey,et al.  Invertibility of linear time-invariant dynamical systems , 1969 .

[26]  P. Schweitzer.  Perturbation theory and finite Markov chains , 1968, Journal of Applied Probability.

[27]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[28]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .