Ergodic BSDEs Driven by Markov Chains

We consider ergodic backward stochastic differential equations (BSDEs), in a setting where noise is generated by a countable state uniformly ergodic Markov chain. We show that for Lipschitz drivers such that a comparison theorem holds, these equations admit unique solutions. To obtain this result, we show by coupling and splitting techniques that uniform ergodicity estimates of Markov chains are robust to perturbations of the rate matrix and that these perturbations correspond in a natural way to ergodic BSDEs. We then consider applications of this theory to Markov decision problems with a risk-averse average reward criterion.

[1]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[2]  Samuel N. Cohen.  Representing filtration consistent nonlinear expectations as $g$-expectations in general probability spaces , 2011, 1102.5287.

[3]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[4]  Serguei Pergamenchtchikov,et al.  Geometric ergodicity for families of homogeneous Markov chains , 2010, 1002.2341.

[5]  Samuel N. Cohen,et al.  Comparisons for backward stochastic differential equations on Markov chains and related no-arbitrage conditions , 2008, 0810.0055.

[6]  M. Royer,et al.  BSDEs with a random terminal time driven by a monotone generator and their links with PDEs , 2004 .

[7]  E. Zeidler.  Nonlinear functional analysis and its applications , 1988 .

[8]  Xianping Guo,et al.  Drift and monotonicity conditions for continuous-time controlled Markov chains with an average criterion , 2003, IEEE Trans. Autom. Control..

[9]  Robert J. Elliott,et al.  Existence, uniqueness and comparisons for BSDEs in general spaces , 2010 .

[10]  Ying Hu,et al.  Stability of BSDEs with Random Terminal Time and Homogenization of Semilinear Elliptic PDEs , 1998 .

[11]  Gianmario Tessitore,et al.  Ergodic BSDEs under weak dissipative assumptions , 2010, 1004.1571.

[12]  P. Kakumanu,et al.  Nondiscounted Continuous Time Markovian Decision Process with Countable State Space , 1972 .

[13]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[14]  S. Peng,et al.  Filtration-consistent nonlinear expectations and related g-expectations , 2002 .

[15]  Robert J. Elliott,et al.  A general comparison theorem for backward stochastic differential equations , 2010, Advances in Applied Probability.

[16]  Adrien Richou,et al.  Ergodic BSDEs and related PDEs with Neumann boundary conditions , 2008, 0807.1521.

[17]  Ying Hu,et al.  Ergodic BSDEs and Optimal Ergodic Control in Banach Spaces , 2007, SIAM J. Control. Optim..

[18]  E. J. McShane,et al.  On Filippov’s implicit functions lemma , 1967 .

[19]  Robert J. Elliott,et al.  Solutions of Backward Stochastic Differential Equations on Markov Chains , 2008 .

[20]  S. Peng,et al.  Adapted solution of a backward stochastic differential equation , 1990 .

[21]  L. Rogers,et al.  Diffusions, Markov processes, and martingales , 1979 .

[22]  T. Lindvall.  Lectures on the Coupling Method , 1992 .

[23]  Samuel N. Cohen,et al.  On Markovian solutions to Markov Chain BSDEs , 2011, 1111.5739.