The reduction of perturbed Markov generators: an algorithm exposing the role of transient states

A new algorithm for the hierarchical aggregation of singularly perturbed finite-state Markov processes is derived. The approach taken bridges the gap between conceptually simple results for a relatively restricted class of processes and the significantly more complex results for the general case. The critical role played by (almost) transient states is exposed, resulting in a straightforward algorithm for the construction of a sequence of aggregate generators associated with various time scales. These generators together provide a uniform asymptotic approximation of the original probability transition function.

[1]  Kishor S. Trivedi,et al.  An Aggregation Technique for the Transient Analysis of Stiff Markov Chains , 1986, IEEE Transactions on Computers.

[2]  Pierre Semal,et al.  Error Bounds for the Analysis by Decomposition of Non-Negative Matrices , 1983, Computer Performance and Reliability.

[3]  Joe H. Chow,et al.  Singular perturbation and iterative separation of time scales , 1980, Autom..

[4]  A. Willsky,et al.  Hierarchical aggregation of linear systems with multiple time scales , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[5]  Lalit R. Bahl,et al.  A Maximum Likelihood Approach to Continuous Speech Recognition , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Peter C. Doerschuk.  A Markov chain approach to electrocardiogram modeling and analysis , 1985 .

[7]  J. R. Rohlicek,et al.  Time Scale Decomposition: The Role of Scaling in Linear Systems and Transient States in Finite-State Markov Processes , 1985, 1985 American Control Conference.

[8]  Jan Robin Rohlicek.  Aggregation and time scale analysis of perturbed Markov systems , 1987 .

[9]  William J. Stewart,et al.  Iterative aggregation/disaggregation techniques for nearly uncoupled markov chains , 1985, JACM.

[10]  George C. Verghese,et al.  An algebraic approach to analysis and control of time-scales , 1983 .

[11]  A. F. Turbin,et al.  Limit theorems for Markov random evolutions in the scheme of asymptotic state lumping , 1983 .

[12]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[13]  Dragoslav D. Šiljak,et al.  Large-Scale Dynamic Systems: Stability and Structure , 1978 .

[14]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[15]  F. Fisher,et al.  Near-Decomposability, Partition and Aggregation, and the Relevance of Stability Discussions , 1963 .

[16]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..

[17]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[18]  F. Delebecque.  A Reduction Process for Perturbed Markov Chains , 1983 .

[19]  Journal of the Association for Computing Machinery , 1961, Nature.