A multiple-shooting differential dynamic programming algorithm. Part 1: Theory

Abstract Multiple-shooting benefits a wide variety of optimal control algorithms by alleviating large sensitivities present in highly nonlinear problems, improving robustness to initial guesses, and increasing the potential for parallel implementation. In this paper series, motivated by the challenges of optimizing highly sensitive spacecraft trajectories, the multiple-shooting approach is embedded for the first time in the formulation of a Differential Dynamic Programming algorithm. Contrary to traditional nonlinear programming methods which necessitate minimal modification of the formulation in order to incorporate multiple-shooting principles, DDP requires novel non-trivial derivations, in order to include the initial conditions of the multiple-shooting subintervals and track their sensitivities. The initial conditions are updated in a necessary additional algorithmic step. A null-space trust-region method is used for the solution of the quadratic subproblems, and equality and inequality path and terminal constraints are handled through a general augmented Lagrangian approach. The propagation of the trajectory and the optimization of its controls are decoupled through the use of State-Transition Matrices. Part 1 of the paper series provides the necessary theoretical developments and implementation details for the algorithm. Part 2 contains validation and application cases.

[1]  Nicholas I. M. Gould,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[2]  Roger Fletcher,et al.  Nonlinear programming and nonsmooth optimization by successive linear programming , 1989, Math. Program..

[3]  W. Murray,et al.  Analytical expressions for the eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions , 1971 .

[4]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[5]  David G. Hull,et al.  Optimal Control Theory for Applications , 2003 .

[6]  Ryan P. Russell,et al.  A Hybrid Differential Dynamic Programming Algorithm for Constrained Optimal Control Problems. Part 1: Theory , 2012, Journal of Optimization Theory and Applications.

[7]  R. Tyrrell Rockafellar,et al.  A dual approach to solving nonlinear programming problems by unconstrained optimization , 1973, Math. Program..

[8]  Nicholas I. M. Gould,et al.  Convergence Properties of an Augmented Lagrangian Algorithm for Optimization with a Combination of General Equality and Linear Constraints , 1996, SIAM J. Optim..

[9]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[10]  J. Betts.  Survey of Numerical Methods for Trajectory Optimization , 1998 .

[11]  S. Yakowitz,et al.  Computational aspects of discrete-time optimal control , 1984 .

[12]  Bruce A. Conway,et al.  A Survey of Methods Available for the Numerical Optimization of Continuous Dynamic Systems , 2011, Journal of Optimization Theory and Applications.

[13]  R. Courant.  Variational methods for the solution of problems of equilibrium and vibrations , 1943 .

[14]  G. J. Whiffen,et al.  Application of a novel optimal control algorithm to low-thrust trajectory optimization , 2001 .

[15]  George Leitmann,et al.  The Calculus of Variations and Optimal Control , 1982 .

[16]  E. Polak,et al.  A globally convergent, implementable multiplier method with automatic penalty limitation , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[17]  P. Deuflhard,et al.  Recent Advances in Multiple Shooting Techniques , 1980 .

[18]  H. J. Pesch.  A Practical Guide to the Solution of Real-Life Optimal Control Problems , 1994 .

[19]  R. Bellman.  Dynamic programming. , 1957, Science.

[20]  Ryan P. Russell,et al.  A Hybrid Differential Dynamic Programming Algorithm for Constrained Optimal Control Problems. Part 2: Application , 2012, Journal of Optimization Theory and Applications.

[21]  Walter Murray,et al.  Sequential Quadratic Programming Methods for Large-Scale Problems , 1997, Comput. Optim. Appl..

[22]  J. B. Rosen.  The gradient projection method for nonlinear programming: Part II , 1961 .

[23]  H. Keller.  Numerical Methods for Two-Point Boundary-Value Problems , 1993 .

[24]  Dieter Kraft,et al.  On Converting Optimal Control Problems into Nonlinear Programming Problems , 1985 .

[25]  Oskar von Stryk,et al.  Direct and indirect methods for trajectory optimization , 1992, Ann. Oper. Res..

[26]  J. Arora,et al.  Differential dynamic programming technique for constrained optimal control , 1991 .

[27]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[28]  Anil V. Rao,et al.  Dichotomic basis approach to solving hyper-sensitive optimal control problems , 1999, Autom..

[29]  J. Betts,et al.  Solving the optimal control problem using a nonlinear programming technique. I - General formulation , 1984 .

[30]  R. Callies,et al.  Optimal Design of a Mission to Neptune , 1993 .

[31]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[32]  R. Tapia.  Diagonalized multiplier methods and quasi-Newton methods for constrained optimization , 1977 .

[33]  P. Deuflhard,et al.  A modified continuation method for the numerical solution of nonlinear two-point boundary value problems by shooting techniques , 1976 .

[34]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[35]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[36]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[37]  Gregory J. Whiffen,et al.  The Jupiter Icy Moons Orbiter reference trajectory , 2006 .

[38]  H. J. Pesch,et al.  Abort landing in windshear: Optimal control problem with third-order state constraint and varied switching structure , 1995 .

[39]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[40]  G. R. Sullivan,et al.  The development of an efficient optimal control package , 1978 .

[41]  Sidney Yakowitz,et al.  Algorithms and Computational Techniques in Differential Dynamic Programming , 1989 .

[42]  D. Jacobson,et al.  A discrete-time differential dynamic programming algorithm with application to optimal orbit transfer , 1970 .

[43]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[44]  M. R. Osborne.  On shooting methods for boundary value problems , 1969 .

[45]  José Mario Martínez,et al.  Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems , 2005, Comput. Optim. Appl..

[46]  Thierry Dargent,et al.  Using Multicomplex Variables for Automatic Computation of High-Order Derivatives , 2010, TOMS.

[47]  J. Betts,et al.  Application of sparse nonlinear programming to trajectory optimization , 1992 .

[48]  Rolf Rannacher,et al.  Indirect Multiple Shooting for Nonlinear Parabolic Optimal Control Problems with Control Constraints , 2014, SIAM J. Sci. Comput..

[49]  Hans Josef Pesch,et al.  Abort landing in the presence of windshear as a minimax optimal control problem, part 2: Multiple shooting and homotopy , 1991 .

[50]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[51]  D. J. Ruxton.  Differential dynamic programming applied to continuous optimal control problems with state variable inequality constraints , 1993 .

[52]  D. Hull.  Conversion of optimal control problems into parameter optimization problems , 1996 .

[53]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[54]  R. Rockafellar.  The multiplier method of Hestenes and Powell applied to convex programming , 1973 .

[55]  J. Betts,et al.  Trajectory optimization on a parallel processor , 1989 .

[56]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[57]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[58]  Philip E. Gill,et al.  Practical optimization , 1981 .

[59]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[60]  P. Toint,et al.  A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds , 1991 .

[61]  H. J. Pesch,et al.  Minimizing the maximum heating of a re-entering space shuttle : An optimal control problem with multiple control constraints , 1996 .

[62]  Anil V. Rao,et al.  Trajectory Optimization: A Survey , 2014 .

[63]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[64]  R. Russell,et al.  On the Computation and Accuracy of Trajectory State Transition Matrices , 2016 .

[65]  P. Deuflhard,et al.  Multiple Shooting Techniques Revisited , 1983 .

[66]  Gregory J. Whiffen,et al.  Mystic: Implementation of the Static Dynamic Optimal Control Algorithm for High-Fidelity, Low-Thrust Trajectory Design , 2006 .

[67]  Pierre-Brice Wieber,et al.  Fast Direct Multiple Shooting Algorithms for Optimal Robot Control , 2005 .

[68]  D. Mayne.  A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[69]  Etienne Pellegrini,et al.  Quasi-Newton Dierential Dynamic Programming for Robust Low-Thrust Optimization , 2012 .