Optimal control of affine nonlinear discrete-time systems

In this paper, direct neural dynamic programming techniques are utilized to solve the Hamilton Jacobi-Bellman equation in real time for the optimal control of general affine nonlinear discrete-time systems. In the presence of partially unknown dynamics, the optimal regulation control problem is addressed while the optimal tracking control problem is addressed in the presence of known dynamics. Each design entails two portions: an action neural network (NN) that is designed to produce a nearly optimal control signal, and a critic NN which evaluates the performance of the system. Novel weight update laws for the critic and action NN's are derived, and all parameters are tuned online. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded (UUB) and that the output of the action NN approaches the optimal control input with small bounded error. Simulation results are also presented to demonstrate the effectiveness of the approach.

[1]  W. Rudin.  Principles of mathematical analysis , 1964 .

[2]  Radoslaw Romuald Zakrzewski,et al.  Neural network control of nonlinear discrete time systems , 1994 .

[3]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[4]  T. Basar,et al.  H/sup /spl infin//-optimal tracking control techniques for nonlinear underactuated systems , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[5]  Jennie Si,et al.  Online learning control by association and reinforcement , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[6]  Jennie Si,et al.  Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence) , 2004 .

[7]  T. Başar,et al.  IEEE Decision and Control Conference Sydney , Australia , Dec 2000 H ∞-Optimal Tracking Control Techniques for Nonlinear Underactuated Systems , 2006 .

[8]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.