Journal of Optimization Theory and Applications Approximate Solutions to the Time-invariant Hamilton-jacobi-bellman Equation 1

In this paper we develop a new method to approximate the solution to the Hamilton-JacobiBellman (HJB) equation which arises in optimal control when the plant is modeled by nonlinear dynamics. The approximation is comprised of two steps. First, successive approximation is used to reduce the HJB equation to a sequence of linear partial di erential equations. These equations are then approximated via Galerkin's spectral method. The resulting algorithm has several important advantages over previously reported methods. Namely, the resulting control is in feedback form and its associated region of attraction is well de ned. In addition, all computations are performed o -line and the control can be made arbitrarily close to optimal. Accordingly this paper presents a new tool for designing nonlinear control systems that adhere to a prescribed integral performance criteria.

[1]  H. Piaggio.  Mathematical Analysis , 1955, Nature.

[2]  L. Kantorovich,et al.  Approximate methods of higher analysis , 1960 .

[3]  Z. Rekasius,et al.  Suboptimal design of intentionally nonlinear controllers , 1964 .

[4]  W. Petryshyn.  On a class of K-p.d. and non-K-p.d. operators and operator equations , 1965 .

[5]  K. Fu,et al.  Feedback Control Which Preserves Optimality for Systems with Unknown Parameters , 1966 .

[6]  P. Brunovský.  On optimal stabilization of nonlinear systems , 1967 .

[7]  Ruey-Wen Liu,et al.  Construction of Suboptimal Control Sequences , 1967 .

[8]  D. Kleinman.  On an iterative technique for Riccati equation computations , 1968 .

[9]  D. Lukes.  Optimal Regulation of Nonlinear Dynamical Systems , 1969 .

[10]  S. G. Mikhlin,et al.  Approximate methods for solution of differential and integral equations , 1970 .

[11]  D. Kleinman,et al.  An easy way to stabilize a linear constant system , 1970 .

[12]  Y. Nishikawa,et al.  A method for suboptimal design of nonlinear feedback systems , 1971 .

[13]  B. Anderson,et al.  Linear Optimal Control , 1971 .

[14]  K. Park.  An approximate method for the synthesis of optimal control of distributed systems , 1972, CDC 1972.

[15]  W. Ames.  The Method of Weighted Residuals and Variational Principles. By B. A. Finlayson. Academic Press, 1972. 412 pp. $22.50. , 1973, Journal of Fluid Mechanics.

[16]  W. E. Bosarge,et al.  The Ritz–Galerkin Procedure for Nonlinear Control Problems , 1973 .

[17]  Jr. N. Sandell.  On Newton's method for Riccati equation solution , 1974 .

[18]  R. Hämäläinen,et al.  On the nonlinear regulator problem , 1975 .

[19]  Dimitri P. Bertsekas,et al.  On error bounds for successive approximation methods , 1976 .

[20]  William L. Garrard,et al.  Design of nonlinear automatic flight control systems , 1977, Autom..

[21]  E. Mageirou.  Iterative techniques for Riccati game equations , 1977 .

[22]  G. Gopalakrishnan Nair,et al.  Suboptimal control of nonlinear systems , 1978, Autom..

[23]  G. Siouris,et al.  Optimum systems control , 1979, Proceedings of the IEEE.

[24]  George N. Saridis,et al.  An Approximation Theory of Optimal Control for Trainable Manipulators , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[25]  J. Tsitsiklis,et al.  Guaranteed robustness properties of multivariable, nonlinear, stochastic optimal regulators , 1983, The 22nd IEEE Conference on Decision and Control.

[26]  I. Dolcetta.  On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming , 1983 .

[27]  R. Marino.  An example of a nonlinear regulator , 1984 .

[28]  E. Ryan.  Optimal feedback control of bilinear systems , 1984 .

[29]  W. A. Cebuhar,et al.  Approximation procedures for the optimal control of bilinear and nonlinear systems , 1984 .

[30]  S. Tzafestas,et al.  Stabilizing optimal control of bilinear systems with a generalized cost , 1984 .

[31]  H. Ishii,et al.  Approximate solutions of the bellman equation of deterministic control theory , 1984 .

[32]  R. González,et al.  On Deterministic Control Problems: An Approximation Procedure for the Optimal Cost I. The Stationary Problem , 1985 .

[33]  T. Glad.  Robust Nonlinear Regulators based on Hamilton-Jacobi Theory and Lyapunov Functions , 1985 .

[34]  Torkel Glad.  Robustness of Nonlinear State Feedback , 1985 .

[35]  R. Bellman,et al.  The Riccati Equation , 1986 .

[36]  W. Baumann,et al.  Feedback control of nonlinear systems by extended linearization , 1986 .

[37]  S. Torkel Glad.  Robustness of nonlinear state feedback - A survey , 1987, Autom..

[38]  M. Falcone.  A numerical approach to the infinite horizon problem of deterministic control theory , 1987 .

[39]  B. Tibken,et al.  An iterative method for the finite-time bilinear-quadratic control problem , 1988 .

[40]  E. Zeidler.  Nonlinear Functional Analysis and Its Applications: II/ A: Linear Monotone Operators , 1989 .

[41]  M. Falcone,et al.  Discrete Dynamic Programming and Viscosity Solutions of the Bellman Equation , 1989 .

[42]  Rolf Johansson,et al.  Quadratic Optimization of Motion Coordination and Control , 1990, 1990 American Control Conference.

[43]  A. Laub.  Invariant Subspace Methods for the Numerical Solution of Riccati Equations , 1991 .

[44]  Philippe Blanchard,et al.  Variational Methods in Mathematical Physics , 1992 .

[45]  R. Luus,et al.  Global optimization approach to nonlinear optimal control , 1992 .

[46]  Lin Chen,et al.  A nonlinear control design for power systems , 1992, Autom..

[47]  G. Saridis,et al.  Suboptimal control for nonlinear stochastic systems , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[48]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[49]  David J. Hill,et al.  Transient stability enhancement and voltage regulation of power systems , 1993 .

[50]  C. J. Goh,et al.  On the nonlinear optimal regulator problem , 1993, Autom..

[51]  H. Kaufman,et al.  Stabilizing a multimachine power system via decentralized feedback linearizing excitation control , 1993 .

[52]  Marija D. Ilic,et al.  Feedback linearizing excitation control on a full-scale power system model , 1994 .

[53]  Marizio Falcone,et al.  Discrete time high-order schemes for viscosity solutions of Hamilton-Jacobi-Bellman equations , 1994 .

[54]  Youyi Wang,et al.  Transient stabilization of power systems with an adaptive control law , 1994, Autom..

[55]  Z. Gajic,et al.  The successive approximation procedure for finite-time optimal control of bilinear systems , 1994, IEEE Trans. Autom. Control..

[56]  R.A. Freeman,et al.  Optimal nonlinear controllers for feedback linearizable systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[57]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[58]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[59]  Emanuel Todorov,et al.  Optimal Control Theory , 2006 .

[60]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.