An historical survey of computational methods in optimal control.

The last twenty years have seen a great flourishing in optimal control theory. In this paper, we shall highlight some of the salient theoretical developments in the specific area of algorithms.The first algorithms for optimal control were aimed at unconstrained problems and were derived by using first and second variation methods of the calculus of variations. These methods have subsequently been recognized as gradient, Newton–Raphson, or Gauss–Newton methods in function space. A much more recent addition to the arsenal of unconstrained optimal control algorithms are several variations of conjugate gradient methods.At first, constrained optimal control problems could only be solved by exterior penalty function methods. Later algorithms specifically designed for constrained problems have appeared. Among these we find methods for solving the unconstrained linear quadratic regulator problem as well as certain constrained minimum time and minimum energy problems. Differential-dynamic programming was developed...

[1]  H. B. Curry.  The method of steepest descent for non-linear minimization problems , 1944 .

[2]  Billy Ruth LeBouf.  A Generalization of Newton's Method , 1948 .

[3]  Arthur M. Hopkin,et al.  A Phase-Plane Approach to the Compensation of Saturating Servomechanisms , 1951, Transactions of the American Institute of Electrical Engineers.

[4]  L. Kantorovich,et al.  Functional analysis in normed spaces , 1952 .

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  D. Bushaw.  DIFFERENTIAL EQUATIONS WITH A DISCONTINUOUS FORCING TERM , 1953 .

[7]  Irving Bogner,et al.  An investigation of the switching criteria for higher order contactor servomechanisms , 1954, Transactions of the American Institute of Electrical Engineers, Part II: Applications and Industry.

[8]  R. Bartle.  Newton’s method in Banach spaces , 1955 .

[9]  H. Chernoff,et al.  Gradient methods of maximization. , 1955 .

[10]  R. Bellman,et al.  On the “bang-bang” control problem , 1956 .

[11]  J. Breakwell.  The Optimization of Trajectories , 1959 .

[12]  J. G. Corput.  On the method of steepest descent , 1960 .

[13]  L. Neustadt.  Synthesizing time optimal control systems , 1960 .

[14]  J. B. Rosen.  The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints , 1960 .

[15]  Henry J. Kelley,et al.  Gradient Theory of Optimal Flight Paths , 1960 .

[16]  Sheldon S. L. Chang,et al.  Synthesis of Optimum Control Systems , 1961 .

[17]  Charles W. Carroll.  The Created Response Surface Technique for Optimizing Nonlinear, Restrained Systems , 1961 .

[18]  C. Desoer,et al.  A minimal time discrete system , 1961 .

[19]  E. Lee,et al.  Optimal control for nonlinear processes , 1961 .

[20]  J. H. Eaton.  An iterative solution to time-optimal control* , 1962 .

[21]  A. Goldstein.  Cauchy's method of minimization , 1962 .

[22]  Winston L. Nelson.  Optimal Control Methods for On-Off Sampling Systems , 1962 .

[23]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[24]  E. Polak,et al.  Minimum Time Control of Second Order Pulse-Width-Modulated Sampled-Data Systems , 1962 .

[25]  Terence Butler,et al.  On a Method of Courant for Minimizing Functionals , 1962 .

[26]  H. Kelley.  Method of Gradients , 1962 .

[27]  J. Warga.  Relaxed variational problems , 1962 .

[28]  H. Kelley.  Guidance theory and extremal fields , 1962 .

[29]  B. Paiewonsky.  Time Optimal Control of Linear Systems with Bounded Controls , 1963 .

[30]  H. G. Moyer,et al.  A trajectory optimization technique based upon the theory of the second variation. , 1963 .

[31]  T. Ważewski.  On an optimal control problem , 1963 .

[32]  P. E. Sarachik,et al.  An Application of Functional Analysis to the Optimal Control Problem , 1963 .

[33]  H. Kelley.  An optimal guidance approximation theory , 1964 .

[34]  Anthony V. Fiacco,et al.  The Sequential Unconstrained Minimization Technique for Nonlinear Programing, a Primal-Dual Method , 1964 .

[35]  H. Knudsen.  An iterative procedure for computing time-optimal controls , 1964 .

[36]  A. Goldstein.  On Newton's method , 1965 .

[37]  A. Bryson,et al.  A SUCCESSIVE SWEEP METHOD FOR SOLVING OPTIMAL PROGRAMMING PROBLEMS , 1965 .

[38]  J. Warga.  Unilateral variational problems with several inequalities. , 1965 .

[39]  R. McGill.  Optimum Control, Inequality State Constraints, and the Generalized Newton-Raphson Algorithm , 1965 .

[40]  V. Boltyanskii.  Sufficient Conditions for Optimality and the Justification of the Dynamic Programming Method , 1966 .

[41]  M. Athans,et al.  An iterative technique for the computation of time optimal controls. , 1966 .

[42]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[43]  R. O. Barr,et al.  Computation of optimal controls by quadratic programming on convex reachable sets , 1966 .

[44]  R. Bellman.  Dynamic programming. , 1957, Science.

[45]  E. Polak,et al.  Constrained Minimization Problems in Finite-Dimensional Spaces , 1966 .

[46]  D. Mayne.  A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[47]  J. B. Rosen.  Optimal control and convex programming. , 1966 .

[48]  R. Bellman,et al.  Quasilinearization and nonlinear boundary-value problems , 1966 .

[49]  Bùi-Trong-Liêu,et al.  La mèthode des centres dans un espace topologique , 1966 .

[50]  J. F. Price,et al.  An effective algorithm for minimization , 1967 .

[51]  G. Franklin,et al.  A second-order feedback method for optimal control computations , 1967, IEEE Transactions on Automatic Control.

[52]  L. Lasdon,et al.  An interior penalty method for inequality constrained optimal control problems , 1967, IEEE Transactions on Automatic Control.

[53]  R. Bucy.  Two-Point Boundary Value Problems of Linear Hamiltonian Systems , 1967 .

[54]  J. B. Rosen,et al.  Solution of Nonlinear Two-Point Boundary Value Problems by Linear Programming , 1967 .

[55]  S. Mcreynolds.  The successive sweep method and dynamic programming , 1967 .

[56]  V. F. Dem'yanov,et al.  The Minimization of a Smooth Convex Functional on a Convex Set , 1967 .

[57]  Allen A. Goldstein,et al.  Constructive Real Analysis , 1967 .

[58]  A. F. Veinott,et al.  On the Convergence of Some Feasible Direction Algorithms for Nonlinear Programming , 1967 .

[59]  J. Daniel.  Convergence of the conjugate gradient method with computationally convenient modifications , 1967 .

[60]  A. Ostrowski.  Contributions to the theory of the method of steepest descent , 1967 .

[61]  S. Mitter,et al.  The conjugate gradient method for optimal control problems , 1967 .

[62]  Robert E. Larson,et al.  State increment dynamic programming , 1968 .

[63]  D. Jacobson.  Second-order and Second-variation Methods for Determining Optimal Control: A Comparative Study using Differential Dynamic Programming† , 1968 .

[64]  A. Balakrishnan.  On a new computing technique in optimal control theory and the maximum principle. , 1968, Proceedings of the National Academy of Sciences of the United States of America.

[65]  H. Sorenson.  Comparison of some conjugate direction procedures for function minimization , 1969 .

[66]  E. Polak.  On primal and dual methods for solving discrete optimal control problems , 1969 .

[67]  E. Polak,et al.  On the convergence of optimization algorithms , 1969 .

[68]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[69]  J. Dieudonne.  Foundations of Modern Analysis , 1969 .

[70]  G. McCormick,et al.  A Generalization of the Method of Balakrishnan: Inequality Constraints and Initial Conditions , 1970 .

[71]  S. Julio.  NUMERICAL SOLUTION OF DYNAMICAL OPTIMIZATION PROBLEMS , 1970 .

[72]  B. N. Pshenichnyi.  Newton's method for the solution of systems of equalities and inequalities , 1970 .

[73]  E. Polak.  On the Implementation of Conceptual Algorithms , 1970 .

[74]  J. Cullum.  An explicit procedure for discretizing continuous, optimal control problems , 1971 .

[75]  E. Polak,et al.  Efficient Implementations of the Polak–Ribière Conjugate Gradient Algorithm , 1972 .

[76]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.