Markov chain approximations for deterministic control problems with affine dynamics and quadratic cost in the control

We restrict our attention to an important class of deterministic optimal control problems: those with dynamics that are affine in the control and cost that is quadratic in the control. As a representative control problem in this class we consider an escape time problem. The chains we construct are much more broadly applicable, since the only germane part of the problem formulation will be the form of the dynamics and the running cost. In particular, these chains can also be used for discounted problems, infinite time problems that do not involve discounting and constrained dynamics.

[1]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[2]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[3]  Harold J. Kushner,et al.  Stochastic systems with small noise, analysis and simulation; a phase locked loop example , 1985 .

[4]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[5]  A numerical method for a calculus of variations problem with discontinuous integrand , 1992 .

[6]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[7]  M. James.  Controlled markov processes and viscosity solutions , 1994 .

[8]  P. Dupuis,et al.  An Optimal Control Formulation and Related Numerical Methods for a Problem in Shape Reconstruction , 1994 .

[9]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[10]  J. Quadrat.  Numerical methods for stochastic control problems in continuous time , 1994 .

[11]  H. Kushner.  Control of Trunk Line Systems in Heavy Traffic , 1995 .

[12]  M. James,et al.  Nonlinear state estimation for uncertain systems with an integral constraint , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[13]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[14]  W. McEneaney.  Robust/ H ∞ filtering for nonlinear systems , 1998 .

[15]  P. Dupuis,et al.  Rates of Convergence for Approximation Schemes in Optimal Control , 1998 .

[16]  Ian R. Petersen,et al.  Nonlinear state estimation for uncertain systems with an integral constraint , 1998, IEEE Trans. Signal Process..

[17]  P. Dupuis,et al.  Markov Chain Approximations for Deterministic Control Problems with Affine Dynamics and Quadratic Cost in the Control , 1999 .