Connections between stochastic control and dynamic games

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous- and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.

[1]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[2]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  P. Whittle.  Risk-sensitive linear/quadratic/gaussian control , 1981, Advances in Applied Probability.

[4]  A. Bensoussan,et al.  Optimal control of partially observable stochastic systems with an exponential-of-integral performance index , 1985 .

[5]  E. Barron,et al.  Total risk aversion, stochastic optimal control, and differential games , 1989 .

[6]  P. Whittle.  Risk-Sensitive Optimal Control , 1990 .

[7]  P. Whittle.  A risk-sensitive maximum principle: the case of imperfect state observation , 1991 .

[8]  Matthew R. James,et al.  Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games , 1992, Math. Control. Signals Syst..

[9]  W. Fleming,et al.  Risk sensitive optimal control and differential games , 1992 .

[10]  M. James,et al.  Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems , 1994, IEEE Trans. Autom. Control..

[11]  T. Runolfsson.  The equivalence between infinite-horizon optimal control of stochastic systems with exponential-of-integral performance index and stochastic differential games , 1994, IEEE Trans. Autom. Control..

[12]  S.I. Marcus,et al.  Risk-sensitive optimal control of hidden Markov models: a case study , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[13]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[14]  C. Charalambous.  The role of information state and adjoint in relating nonlinear output feedback risk-sensitive control and dynamic games , 1997, IEEE Trans. Autom. Control..

[15]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .