Verifiable Conditions for Irreducibility, Aperiodicity and T-chain Property of a General Markov Chain

We consider in this paper Markov chains on a state space being an open subset of R n that obey the following general non linear state space model: Φt+1 = F (Φt, α(Φt, Ut+1)) , t ∈ N, where (Ut) t∈N * (each Ut ∈ R p) are i.i.d. random vectors, the function α, taking values in R m , is a measurable typically discontinuous function and (x, w) → F (x, w) is a C 1 function. In the spirit of the results presented in the chapter 7 of the Meyn and Tweedie book on " Markov Chains and Stochastic Stability " , we use the underlying deterministic control model to provide sufficient conditions that imply that the chain is a ϕ-irreducible, aperiodic T-chain with the support of the maximal irreducibility measure that has a non empty interior. To be able to show the same properties using previous results would require that the overall update function (x, u) → F (x, α(x, u)) is C ∞ and that U1 admits a lower semi-continuous density. In contrast, we assume that the function (x, w) → F (x, w) is C 1 , and that for all x, α(x, U1) admits a density px such that the function (x, w) → px(w) is lower semi-continuous. Hence the function (x, u) → F (x, α(x, u)) may have discontinuities captured by the function α. We introduce the notion of a strongly globally attracting state and we prove that if there exists a strongly globally attracting state and a time step k, such that we find a k-path such that the k th transition function starting from x * , F k (x * , .), is a submersion at this k-path, the chain is a ϕ-irreducible, aperiodic, T-chain. We present two applications of our results to Markov chains arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.