Risk sensitive control of finite state machines on an infinite horizon. I

Robust and risk sensitive control of discrete time, finite state systems on an infinite horizon is considered, with either state or output feedback. The solution of the state feedback robust control problem is characterized in terms of the value of an average cost dynamic game. The risk sensitive stochastic optimal control problem is solved using the policy iteration algorithm, and the optimal rate is expressed in terms of the value of a stochastic dynamic game with average cost per unit time criterion. By taking a small noise limit a deterministic dynamic game is obtained, which is closely related to the robust control problem. For the problem with output feedback (partial state information) the analysis depends on introducing appropriate information states for the risk sensitive and robust control problems.

[1]  H. Furstenberg,et al.  Products of Random Matrices , 1960 .

[2]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state-information II. The convexity of the lossfunction , 1969 .

[3]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[4]  Thomas Kaijser.  A Limit Theorem for Partially Observed Markov Chains , 1975 .

[5]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[6]  Ari Arapostathis,et al.  Analysis of an identification algorithm arising in the adaptive estimation of Markov chains , 1985, 1985 24th IEEE Conference on Decision and Control.

[7]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[8]  M. James,et al.  Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems , 1994, IEEE Trans. Autom. Control..

[9]  S.I. Marcus,et al.  Risk-sensitive optimal control of hidden Markov models: a case study , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[10]  M. James,et al.  Robust and Risk-Sensitive Output Feedback Control for Finite State Machines and Hidden Markov Models , 1994 .

[11]  L. Stettner,et al.  Approximations of discrete time partially observed control problems , 1994 .

[12]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[13]  M. James,et al.  Robust H∞ output feedback control for nonlinear systems , 1995, IEEE Trans. Autom. Control..

[14]  S. Marcus,et al.  Risk sensitive control of Markov processes in countable state space , 1996 .

[15]  W. Fleming,et al.  Risk-Sensitive Control of Finite State Machines on an Infinite Horizon I , 1997 .

[16]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[17]  E. Fernández-Gaucherand,et al.  Risk-sensitive optimal control of hidden Markov models: structural results , 1997, IEEE Trans. Autom. Control..

[18]  Matthew R. James,et al.  Dissipativity and nonlinear systems with finite power gain , 1998 .