Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems

In this paper we solve a finite-horizon partially observed risk-sensitive stochastic optimal control problem for discrete-time nonlinear systems and obtain small noise and small risk limits. The small noise limit is interpreted as a deterministic partially observed dynamic game, and new insights into the optimal solution of such game problems are obtained. Both the risk-sensitive stochastic control problem and the deterministic dynamic game problem are solved using information states, dynamic programming, and associated separated policies. A certainty equivalence principle is also discussed. Our results have implications for the nonlinear robust stabilization problem. The small risk limit is a standard partially observed risk-neutral stochastic optimal control problem.

[1]  D. Bertsekas,et al.  Sufficiently informative functions and the minimax feedback control of uncertain dynamic systems , 1973 .

[2]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[3]  Willi Hock,et al.  Lecture Notes in Economics and Mathematical Systems , 1981 .

[4]  P. Whittle.  Risk-sensitive linear/quadratic/gaussian control , 1981, Advances in Applied Probability.

[5]  P. Krishnaprasad,et al.  Dynamic observers as asymptotic limits of recursive filters , 1982, 1982 21st IEEE Conference on Decision and Control.

[6]  A. Bensoussan,et al.  Optimal control of partially observable stochastic systems with an exponential-of-integral performance index , 1985 .

[7]  Asymptotic Bayesian Estimation of a First Order Equation with Small Diffusion , 1984 .

[8]  D. Sworder.  Stochastic calculus and applications , 1984, IEEE Transactions on Automatic Control.

[9]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[10]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[11]  M. James,et al.  Nonlinear filtering and large deviations:a pde-control theoretic approach , 1988 .

[12]  K. Glover,et al.  State-space formulae for all stabilizing controllers that satisfy and H ∞ norm bound and relations to risk sensitivity , 1988 .

[13]  P. Whittle.  A risk-sensitive maximum principle , 1990 .

[14]  Matthew R. James.  Finite time observer design by probabilistic-variational methods , 1991 .

[15]  P. Whittle.  A risk-sensitive maximum principle: the case of imperfect state observation , 1991 .

[16]  J. Speyer,et al.  A game-theoretic approach to a finite-time disturbance attenuation problem , 1991 .

[17]  Bo Bernhardsson,et al.  Topics in Digital and Robust Control of Linear Systems , 1992 .

[18]  Matthew R. James,et al.  Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games , 1992, Math. Control. Signals Syst..

[19]  W. Fleming,et al.  Risk sensitive optimal control and differential games , 1992 .

[20]  B. Anderson,et al.  A game theoretic approach to H ∞ control for time-varying systems , 1992 .