Risk-sensitive optimal control of hidden Markov models: a case study

We consider a risk-sensitive optimal control problem for hidden Markov models (HMM), i.e., controlled Markov chains where state information is only available to the controller via an output (message) process. The optimal control of HMM under standard, risk-neutral performance criteria, e.g., discounted and average costs, has received much attention in the past. By reference to a 2-state replacement problem with failure-prone units, it is discussed how risk-sensitivity manifests itself in a controller.<<ETX>>

[1]  M. James,et al.  Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems , 1994, IEEE Trans. Autom. Control..

[2]  Ari Arapostathis,et al.  Analysis of an adaptive control scheme for a partially observed controlled Markov chain , 1990 .

[3]  S.,et al.  Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems , 1994 .

[4]  M. James,et al.  Robust and Risk-Sensitive Output Feedback Control for Finite State Machines and Hidden Markov Models , 1994 .

[5]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[6]  William S. Lovejoy.  Technical Note - On the Convexity of Policy Regions in Partially Observed Systems , 1987, Oper. Res..

[7]  M. J. Sobel,et al.  Discounted MDP's: distribution functions and exponential utility maximization , 1987 .

[8]  A. Bensoussan,et al.  Optimal control of partially observable stochastic systems with an exponential-of-integral performance index , 1985 .

[9]  Ari Arapostathis,et al.  On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes , 1991, Ann. Oper. Res..

[10]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[11]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[12]  Chelsea C. White,et al.  A Markov Quality Control Process Subject to Partial Observation , 1977 .

[13]  R. Howard,et al.  Risk-Sensitive Markov Decision Processes , 1972 .

[14]  A. N. Shiryaev.  On Markov Sufficient Statistics in Non-Additive Bayes Problems of Sequential Analysis , 1964 .

[15]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[16]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..