Optimal tracking control based on reinforcement learning value iteration algorithm for time-delayed nonlinear systems with external disturbances and input constraints

Abstract This article investigates the design of an optimal tracking controller for a class of nonlinear continuous-time systems with time-delay, mismatched external disturbances and input constraints. The technique of integral reinforcement learning (IRL) is utilized for determining the control input that optimizes an objective function. To enable the usage of an estimation of the external disturbances in the recursive objective function, a disturbance observer is designed. For the derivation of the optimal control input, a Hamilton-Jacobi-Bellman (HJB) equation is employed and solved using the iterative IRL algorithm. The proposed approach guarantees that in the presence of mismatched disturbances, the output of the time-delayed nonlinear system tracks the desired trajectory with bounded error. A critic neural network is designed for the implementation of the proposed approach. The efficiency of the method is illustrated by a simulation example.