Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterize the value function via Hamilton Jacobi Bellman equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

[1]  J.H.F. Linssen,et al.  continuous-time markov decision processes , 2016 .

[2]  Y. Kamarianakis.  Ergodic control of diffusion processes , 2013 .

[3]  V. Borkar,et al.  Risk-Sensitive Control with Near Monotone Cost , 2010 .

[4]  Onésimo Hernández-Lerma,et al.  Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains , 2009, Math. Methods Oper. Res..

[5]  Xianping Guo,et al.  Continuous-Time Markov Decision Processes: Theory and Applications , 2009 .

[6]  M. K. Ghosh,et al.  Portfolio Optimization in a Semi-Markov Modulated Market , 2009 .

[7]  Anna Ja 'skiewicz.  Average optimality for risk-sensitive control with general state space , 2007 .

[8]  Xi-Ren Cao,et al.  A survey of recent results on continuous-time Markov decision processes , 2006 .

[9]  Tomasz R. Bielecki,et al.  Risk Sensitive Portfolio Management with Cox--Ingersoll--Ross Interest Rates: The HJB Equation , 2005, SIAM J. Control. Optim..

[10]  J. Menaldi,et al.  Remarks on Risk-Sensitive Control Problems , 2005 .

[11]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[12]  S. Meyn,et al.  Large Deviations Asymptotics and the Spectral Theory of Multiplicatively Regular Markov Processes , 2005, math/0509310.

[13]  S. Meyn,et al.  Spectral theory and limit theorems for geometrically ergodic Markov processes , 2002, The Annals of Applied Probability.

[14]  Sean P. Meyn,et al.  Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost , 2002, Math. Oper. Res..

[15]  Lukasz Stettner,et al.  Risk-Sensitive Control of Discrete-Time Markov Processes with Infinite Horizon , 1999, SIAM J. Control. Optim..

[16]  S. Pliska,et al.  Risk-Sensitive Dynamic Asset Management , 1999 .

[17]  Matthew J. Sobel,et al.  Risk-Sensitive Dynamic Market Share Attraction Games , 1997 .

[18]  Daniel Hernández-Hernández,et al.  Risk Sensitive Markov Decision Processes , 1997 .

[19]  S. Marcus,et al.  Risk sensitive control of Markov processes in countable state space , 1996 .

[20]  H. Nagai.  Bellman equations of risk sensitive control , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[21]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[22]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[23]  O. Hernondex-lerma,et al.  Adaptive Markov Control Processes , 1989 .

[24]  M. J. Sobel,et al.  Discounted MDP's: distribution functions and exponential utility maximization , 1987 .

[25]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[26]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[27]  V. Benes.  Existence of Optimal Strategies Based on Specified Information, for a Class of Stochastic Decision Problems , 1970 .

[28]  W. Sharpe.  CAPITAL ASSET PRICES: A THEORY OF MARKET EQUILIBRIUM UNDER CONDITIONS OF RISK* , 1964 .