The ergodic theory of Markov chains in random environments

SummaryA general formulation of the stochastic model for a Markov chain in a random environment is given, including an analysis of the dependence relations between the environmental process and the controlled Markov chain, in particular the problem of feedback. Assuming stationary environments, the ergodic theory of Markov processes is applied to give conditions for the existence of finite invariant measure (equilibrium distributions) and to obtain ergodic theorems, which provide results on convergence of products of random stochastic matrices. Coupling theory is used to obtain results on direct convergence of these products and the structure of the tail σ-field. State properties including classification and communication properties are discussed.

[1]  S. Kakutani.  Random Ergodic Theorems and Markoff Processes with a Stable Distribution , 1951 .

[2]  J. Doob.  Stochastic processes , 1953 .

[3]  M. Bartlett,et al.  Weak ergodicity in non-homogeneous Markov chains , 1958, Mathematical Proceedings of the Cambridge Philosophical Society.

[4]  H. Furstenberg,et al.  Products of Random Matrices , 1960 .

[5]  Michel Loève,et al.  Probability Theory I , 1977 .

[6]  Yukio Takahashi.  Markov chains with random transition matrices , 1969 .

[7]  Samuel Karlin,et al.  On Branching Processes with Random Environments: I: Extinction Probabilities , 1971 .

[8]  Samuel Karlin,et al.  Branching Processes with Random Environments, II: Limit Theorems , 1971 .

[9]  J. F. C. Kingman,et al.  The ergodic theory of Markov processes , 1971, The Mathematical Gazette.

[10]  D. Griffeath.  A maximal coupling for Markov chains , 1975 .

[11]  Ein zufälliger Ergodensatz für eine Familie stochastischer Matrizen ohne gemeinsames invariantes Verteilungsgesetz. Zum Gedenken an Wolfgang Richter , 1975 .

[12]  J. Cohen.  Ergodicity of Age Structure in Populations with Markovian Vital Rates, I: Countable States , 1976 .

[13]  Partial Coupling and Loss of Memory for Markov Chains , 1976 .

[14]  J. Hajnal,et al.  On products of non-negative matrices , 1976, Mathematical Proceedings of the Cambridge Philosophical Society.

[15]  Joel E. Cohen,et al.  Ergodicity of age structure in populations with Markovian vital rates, III: Finite-state moments and growth rate; an illustration , 1977, Advances in Applied Probability.

[16]  J. Cohen.  Ergodicity of age structure in populations with Markovian vital rates. II. General states , 1977, Advances in Applied Probability.

[17]  Ein Grenzwertsatz fr stationre zufllige Folgen stochastischer Matrizen , 1977 .

[18]  Joel E. Cohen,et al.  RANDOM EVOLUTIONS IN DISCRETE AND CONTINUOUS TIME , 1979 .

[19]  Joel E. Cohen,et al.  Ergodic theorems in demography , 1979 .

[20]  R. Cogburn,et al.  Markov Chains in Random Environments: The Case of Markovian Environments , 1980 .

[21]  Kurt Nawrotzki.  Stationäre Verteilungen endlicher offener Systeme , 1980, J. Inf. Process. Cybern..

[22]  Random products of stochastic matrices , 1981 .

[23]  Birth and death processes with random environments in continuous time. , 1981, Journal of applied probability.

[24]  ON DETERMINING ABSORPTION PROBABILITIES FOR MARKOV CHAINS IN RANDOM ENVIRONMENTS , 1981 .

[25]  K. Nawrotzki.  Finite Markov Chains in Stationary Random Environments , 1982 .

[26]  Kurt Nawrotzki,et al.  Discrete Open Systems or Markov Chains in a Random Environment II , 1982, Journal of Information Processing and Cybernetics.