Reinforcement Learning Output Feedback NN Control Using Deterministic Learning Technique

In this brief, a novel adaptive-critic-based neural network (NN) controller is investigated for nonlinear pure-feedback systems. The controller design is based on the transformed predictor form, and the actor-critic NN control architecture includes two NNs, whereas the critic NN is used to approximate the strategic utility function, and the action NN is employed to minimize both the strategic utility function and the tracking error. A deterministic learning technique has been employed to guarantee that the partial persistent excitation condition of internal states is satisfied during tracking control to a periodic reference orbit. The uniformly ultimate boundedness of closed-loop signals is shown via Lyapunov stability analysis. Simulation results are presented to demonstrate the effectiveness of the proposed control.

[1]  Cong Wang,et al.  Deterministic Learning and Rapid Dynamical Pattern Recognition , 2007, IEEE Transactions on Neural Networks.

[2]  George Meyer,et al.  Stable inversion for nonlinear systems , 1997, Autom..

[3]  Cong Wang,et al.  Learning from neural control , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[4]  Shuzhi Sam Ge,et al.  Robust Adaptive Neural Network Control for a Class of Uncertain MIMO Nonlinear Systems With Input Nonlinearities , 2010, IEEE Transactions on Neural Networks.

[5]  Derong Liu,et al.  Neural-Network-Based Optimal Control for a Class of Unknown Discrete-Time Nonlinear Systems Using Globalized Dual Heuristic Programming , 2012, IEEE Transactions on Automation Science and Engineering.

[6]  Yi Zhang,et al.  A self-learning call admission control scheme for CDMA cellular networks , 2005, IEEE Transactions on Neural Networks.

[7]  Qinmin Yang,et al.  Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems using Online Approximators , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Derong Liu,et al.  Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear Systems , 2013, IEEE Transactions on Cybernetics.

[9]  Jean-Jacques E. Slotine,et al.  Neural Network Control of Unknown Nonlinear Systems , 1989, 1989 American Control Conference.

[10]  Junmin Li,et al.  Decentralized Output-Feedback Neural Control for Systems With Unknown Interconnections , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[12]  Shuzhi Sam Ge,et al.  An ISS-modular approach for adaptive neural control of pure-feedback systems , 2006, Autom..

[13]  Shuzhi Sam Ge,et al.  Output Feedback NN Control for Two Classes of Discrete-Time Systems With Unknown Control Directions in a Unified Approach , 2008, IEEE Transactions on Neural Networks.

[14]  Guo-Xing Wen,et al.  Direct adaptive NN control for a class of discrete-time nonlinear strict-feedback systems , 2010, Neurocomputing.

[15]  Radoslaw Romuald Zakrzewski,et al.  Neural network control of nonlinear discrete time systems , 1994 .

[16]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[17]  D. Mayne.  Nonlinear and Adaptive Control Design [Book Review] , 1996, IEEE Transactions on Automatic Control.

[18]  Lei Yang,et al.  Direct Heuristic Dynamic Programming for Nonlinear Tracking Control With Filtered Tracking Error , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Licheng Jiao,et al.  Adaptive Tracking for Periodically Time-Varying and Nonlinearly Parameterized Systems Using Multilayer Neural Networks , 2010, IEEE Transactions on Neural Networks.

[20]  Nader Sadegh,et al.  A perceptron network for functional identification and control of nonlinear systems , 1993, IEEE Trans. Neural Networks.

[21]  Cong Wang,et al.  Identification and Learning Control of Ocean Surface Ship Using Neural Networks , 2012, IEEE Transactions on Industrial Informatics.

[22]  Cong Wang,et al.  Learning from neural control for a class of discrete-time nonlinear systems , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[23]  David J. Hill,et al.  Deterministic Learning Theory , 2009 .

[24]  Shuzhi Sam Ge,et al.  Adaptive Predictive Control Using Neural Network for a Class of Pure-Feedback Systems in Discrete Time , 2008, IEEE Transactions on Neural Networks.

[25]  Shaocheng Tong,et al.  Adaptive Neural Output Feedback Tracking Control for a Class of Uncertain Discrete-Time Nonlinear Systems , 2011, IEEE Transactions on Neural Networks.