The Decomposition-Separation Theorem for Finite Nonhomogeneous Markov Chains and Related Problems

Let M be a finite set, P be a stochastic matrix and U = {(Zn)} be the family of all finite Markov chains (MC) (Zn) defined by M , P , and all possible initial distributions. The behavior of a MC (Zn) is a classical result of probability theory derived in the 1930s by A. N. Kolmogorov and W. Doeblin. If a stochastic matrix P is replaced by a sequence of stochastic matrices (Pn) and transitions at moment n are defined by Pn, then U becomes a family of nonhomogeneous MCs. There are numerous results concerning the behavior of such MCs given some specific properties of the sequence (Pn). But what if there are no assumptions about sequence (Pn)? Is it possible to say something about the behavior of the family U? The surprising answer to this question is Yes. Such behavior is described by a theorem which we call a decompositionseparation (DS) theorem, and which was initiated by a small paper of A. N. Kolmogorov (1936) and formulated and proved in a few stages in a series of papers including D. Blackwell (1945), H. Cohn (1971, 1989), and I. Sonin

[1]  M. S. Bartlett,et al.  The ergodic properties of non-homogeneous finite Markov chains , 1956, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  E. Seneta,et al.  On the historical development of the theory of finite inhomogeneous Markov chains , 1973, Mathematical Proceedings of the Cambridge Philosophical Society.

[3]  D. Griffeath.  Uniform coupling of non-homogeneous Markov chains , 1975, Journal of Applied Probability.

[4]  J. F. C. Kingman,et al.  Geometrical aspects of the theory of non-homogeneous Markov chains , 1975, Mathematical Proceedings of the Cambridge Philosophical Society.

[5]  E. Ruch,et al.  The principle of increasing mixing character and some of its consequences , 1976 .

[6]  Anthony Unwin,et al.  Markov Chains — Theory and Applications , 1977 .

[7]  T. Hill.  On the Existence of Good Markov Strategies , 1979 .

[8]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[9]  A. Uhlmann,et al.  Stochasticity and Partial Order: Doubly Stochastic Maps and Unitary Mixing , 1982 .

[10]  Non-homogeneous markov chains: Tail idempotents, tail sigma-fields and basis , 1983 .

[11]  A theorem on separation of jets and some properties of random sequences , 1987 .

[12]  The separation of jets and some asymptotic properties of random sequences , 1988 .

[13]  H. Cohn.  PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS , 1989 .

[14]  I. M. Sonin.  On an extremal property of Markov chains and sufficiency of Markov strategies in Markov decision processes with the Dubins-Savage criterion , 1991, Ann. Oper. Res..

[15]  I. Sonin.  An Arbitrary Nonhomogeneous Markov Chain with Bounded Number of States May Be Decomposed into Asymptotically Noncommunicating Components Having the Mixing Property , 1992 .

[16]  P. Pokarowski,et al.  Tail events of some nonhomogeneous Markov chains , 1995 .

[17]  I. Sonin.  The asymptotic behaviour of a general finite nonhomogeneous Markov chain (the decomposition-separation theorem) , 1996 .

[18]  L. Wen,et al.  An extension of Shannon-McMillan theorem and some limit properties for nonhomogeneous Markov chains , 1996 .

[19]  H. Cohn,et al.  Simulated Annealing: Searching for an Optimal Temperature Schedule , 1999, SIAM J. Optim..

[20]  H. Thorisson.  Coupling, stationarity, and regeneration , 2000 .

[21]  Sunder Sethuraman,et al.  A Martingale Proof of Dobrushin's Theorem for Non-Homogeneous Markov Chains , 2004 .

[22]  Sunder Sethuraman,et al.  Large deviations for a class of nonhomogeneous Markov chains , 2005 .

[23]  B. Nordstrom.  FINITE MARKOV CHAINS , 2005 .

[24]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[25]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..