On infinite products of stochastic matrices

For a sequence of stochastic matrices {Qk}∞k=0 we establish conditions for weak ergodicity of products taken in an arbitrary order of multiplication, and strong ergodicity (i.e. convergence) of the backward products MN=QNQN−1⋯Q1Q0. Such a condition is, e.g., that at least one of the limit points of {Qk}∞k=0 is scrambling. We also study strong ergodicity for products taken in an arbitrary order of multiplication. This holds, e.g., if {Qk}∞k=0 converges to a regular matrix, or when all the limit points of {Qk}∞k=0 are doubly stochastic and scrambling. We describe two applications. First, we establish the existence of optimal strategies for controlled Markov chains on infinite horizon. Second, we show that the growth properties of certain linear nonautonomous differential equations are determined by the projection of the initial value on a certain characteristic direction.