A new approach to differential dynamic programming for discrete time systems

This paper proposes a new differential dynamic programming algorithm for solving discrete time optimal control problems with equality and inequality constraints on both control and state variables and proves its convergence. The present algorithm is different from differential dynamic programming algorithms developed in [10]-[15], which can hardly solve optimal control problems with inequality constraints on state variables and whose convergence has not been proved. Composed of iterative methods for solving systems of nonlinear equations, it is based upon Kuhn-Tucker conditions for recurrence relations of dynamic programming. Numerical examples show file efficiency of the present algorithm.

[1]  R. E. Kalman,et al.  Control System Analysis and Design Via the “Second Method” of Lyapunov: I—Continuous-Time Systems , 1960 .

[2]  R. E. Kalman,et al.  Control System Analysis and Design Via the “Second Method” of Lyapunov: II—Discrete-Time Systems , 1960 .

[3]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[4]  D. Mayne.  A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[5]  D. Jacobson,et al.  A transformation technique for optimal control problems with a state variable inequality constraint , 1969 .

[6]  K. H. Well,et al.  Modified quasilinearization and optimal initial choice of the multipliers part 2—Optimal control problems , 1970 .

[7]  D. Jacobson,et al.  A discrete-time differential dynamic programming algorithm with application to optimal orbit transfer , 1970 .

[8]  Elijah Polak,et al.  A survey of methods of feasible directions for the solution of optimal control problems , 1972 .

[9]  R. Mehra,et al.  A generalized gradient method for optimal control problems with inequality constraints and singular arcs , 1972 .

[10]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[11]  R. Havira,et al.  Computation of quantized controls using differential dynamic programming , 1972 .

[12]  E. Polak.  An historical survey of computational methods in optimal control. , 1973 .

[13]  David Q. Mayne,et al.  Differential Dynamic Programming–A Unified Approach to the Optimization of Dynamic Systems , 1973 .

[14]  E. Polak,et al.  NEWTON DERIVED METHODS FOR NONLINEAR EQUATIONS AND INEQUALITIES , 1975 .

[15]  D. Mayne,et al.  First-order strong variation algorithms for optimal control , 1975 .

[16]  Ole H. Hald.  On a Newton-Moser type method , 1975 .

[17]  D. Mayne,et al.  First-order strong variation algorithms for optimal control problems with terminal inequality constraints , 1975 .