Fuzzy-Based Goal Representation Adaptive Dynamic Programming

In this paper, a novel nonlinear learning controller called fuzzy-based goal representation adaptive dynamic programming (Fuzzy-GrADP) is proposed. In the proposed GrADP method, a goal representation network is introduced to generate an adaptive internal reinforcement signal to the critic network to help the controller provide a general mapping between the input and output actions. Moreover, in the proposed architecture, the action network in the GrADP is improved by using the fuzzy hyperbolic model, which combines the merits of the fuzzy model and the neural network model. Based on the back-propagation technique, the parameters in the membership functions and the fuzzy rules are all undergo training and online adapting. The proposed controller is tested on two numerical benchmarks, and the simulation results show that the proposed controller outperforms the original adaptive dynamic fuzzy controller and the pure neural network-based GrADP controller. In addition, the proposed controller is further applied on a large multimachine power system for static var compensator damping control, where simulation results demonstrate the effectiveness of the proposed approach on real applications. Furthermore, in order to demonstrate the theoretical guarantee of the proposed method, Lyapunov stability analysis to support the proposed Fuzzy-GrADP approach has also been carried out.

[1]  Haibo He,et al.  Reinforcement learning control based on multi-goal representation using hierarchical heuristic dynamic programming , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[2]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[3]  Kazuo Tanaka,et al.  An approach to fuzzy control of nonlinear systems: stability and design issues , 1996, IEEE Trans. Fuzzy Syst..

[4]  Tsung-Chih Lin,et al.  Direct adaptive fuzzy-neural control with state observer and supervisory controller for unknown nonlinear dynamical systems , 2002, IEEE Trans. Fuzzy Syst..

[5]  Jinyu Wen,et al.  Wide-Area Damping Controller for Power System Interarea Oscillations: A Networked Predictive Control Approach , 2015, IEEE Transactions on Control Systems Technology.

[6]  George G. Lendaris,et al.  Adaptive critic based approximate dynamic programming for tuning fuzzy controllers , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[7]  Gang Wang,et al.  Fuzzy hyperbolic neural network with time-varying delays , 2010, Fuzzy Sets Syst..

[8]  Qinglai Wei,et al.  Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming , 2012, Autom..

[9]  Jinyu Wen,et al.  Design of Anti-Windup Compensator for Energy Storage-Based Damping Controller to Enhance Power System Stability , 2014, IEEE Transactions on Power Systems.

[10]  Chin-Teng Lin,et al.  Reinforcement structure/parameter learning for neural-network-based fuzzy logic control systems , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[11]  Donald C. Wunsch,et al.  Neurocontroller alternatives for "fuzzy" ball-and-beam systems with nonuniform nonlinear friction , 2000, IEEE Trans. Neural Networks Learn. Syst..

[12]  Shie-Jue Lee,et al.  A neuro-fuzzy system modeling with self-constructing rule generationand hybrid SVD-based learning , 2003, IEEE Trans. Fuzzy Syst..

[13]  Dongbin Zhao,et al.  Integration of fuzzy controller with adaptive dynamic programming , 2012, Proceedings of the 10th World Congress on Intelligent Control and Automation.

[14]  Jin Bae Park,et al.  Neural-Network-Based Decentralized Adaptive Control for a Class of Large-Scale Nonlinear Systems With Unknown Time-Varying Delays , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Haibo He,et al.  Reactive power control of grid-connected wind farm based on adaptive dynamic programming , 2014, Neurocomputing.

[16]  Paul J. Werbos,et al.  2009 Special Issue: Intelligence in the brain: A theory of how it works and how to build it , 2009 .

[17]  Zhou Quan,et al.  RBF Neural Network and ANFIS-Based Short-Term Load Forecasting Approach in Real-Time Price Environment , 2008, IEEE Transactions on Power Systems.

[18]  Guo-Xing Wen,et al.  Fuzzy Neural Network-Based Adaptive Control for a Class of Uncertain Nonlinear Stochastic Systems , 2014, IEEE Transactions on Cybernetics.

[19]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[20]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[21]  Chih-Hong Lin,et al.  Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive , 2001, IEEE Trans. Fuzzy Syst..

[22]  Zhang Huaguang,et al.  Modeling, identification, and control of a class of nonlinear systems , 2001, IEEE Trans. Fuzzy Syst..

[23]  Haibo He,et al.  A three-network architecture for on-line learning and optimization based on adaptive dynamic programming , 2012, Neurocomputing.

[24]  Shaocheng Tong,et al.  Robust Adaptive Tracking Control for Nonlinear Systems Based on Bounds of Fuzzy Approximation Parameters , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[25]  Peihua Qiu,et al.  Fuzzy Modeling and Fuzzy Control , 2006, Technometrics.

[26]  Haibo He,et al.  Neural and fuzzy dynamic programming for under-actuated systems , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[27]  Haibo He,et al.  Comparative study between HDP and PSS on DFIG damping control , 2013, 2013 IEEE Computational Intelligence Applications in Smart Grid (CIASG).

[28]  Chuen-Tsai Sun,et al.  Neuro-fuzzy modeling and control , 1995, Proc. IEEE.

[29]  Huaguang Zhang,et al.  Nearly Optimal Control Scheme Using Adaptive Dynamic Programming Based on Generalized Fuzzy Hyperbolic Model , 2013 .

[30]  Haibo He,et al.  Adaptive control for an HVDC transmission link with FACTS and a wind farm , 2013, 2013 IEEE PES Innovative Smart Grid Technologies Conference (ISGT).

[31]  Jennie Si,et al.  Online learning control by association and reinforcement. , 2001, IEEE transactions on neural networks.

[32]  Jinyu Wen,et al.  Energy-Storage-Based Low-Frequency Oscillation Damping Control Using Particle Swarm Optimization and Heuristic Dynamic Programming , 2014, IEEE Transactions on Power Systems.

[33]  Haibo He,et al.  Power System Stability Control for a Wind Farm Based on Adaptive Dynamic Programming , 2015, IEEE Transactions on Smart Grid.

[34]  Huaguang Zhang,et al.  Modeling, identification, and control of a class of nonlinear systems , 2001, IEEE Trans. Fuzzy Syst..

[35]  Chuen-Chien Lee,et al.  Fuzzy logic in control systems: fuzzy logic controller. II , 1990, IEEE Trans. Syst. Man Cybern..

[36]  Changjiu Zhou,et al.  Adaptive fuzzy H/sub /spl infin// stabilization for strict-feedback canonical nonlinear systems via backstepping and small-gain approach , 2005, IEEE Transactions on Fuzzy Systems.

[37]  Jianbin Qiu,et al.  T–S-Fuzzy-Model-Based Approximation and Controller Design for General Nonlinear Systems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  Wei Yao,et al.  Wide-Area Damping Controller of FACTS Devices for Inter-Area Oscillations Considering Communication Time Delays , 2014, IEEE Transactions on Power Systems.

[39]  Babu Narayanan,et al.  POWER SYSTEM STABILITY AND CONTROL , 2015 .

[40]  K. R. Padiyar,et al.  ENERGY FUNCTION ANALYSIS FOR POWER SYSTEM STABILITY , 1990 .

[41]  Jinyu Wen,et al.  Adaptive Learning in Tracking Control Based on the Dual Critic Network Design , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Li-Xin Wang.  Stable adaptive fuzzy control of nonlinear systems , 1993, IEEE Trans. Fuzzy Syst..

[43]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[44]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[45]  Seetha Hari,et al.  Learning From Imbalanced Data , 2019, Advances in Computer and Electrical Engineering.

[46]  Andrzej Bartoszewicz,et al.  ITAE Optimal Sliding Modes for Third-Order Systems With Input Signal and State Constraints , 2010, IEEE Transactions on Automatic Control.

[47]  Derong Liu,et al.  Neural-Network-Based Optimal Control for a Class of Unknown Discrete-Time Nonlinear Systems Using Globalized Dual Heuristic Programming , 2012, IEEE Transactions on Automation Science and Engineering.

[48]  Haibo He,et al.  Intelligent load frequency controller using GrADP for island smart grid with electric vehicles and renewable resources , 2015, Neurocomputing.

[49]  James D. McCalley,et al.  Damping controller design for power system oscillations using global signals , 1996 .

[50]  Haibo He,et al.  Optimal Control for Unknown Discrete-Time Nonlinear Markov Jump Systems Using Adaptive Dynamic Programming , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Junfei Qiao,et al.  Nonlinear Systems Modeling Based on Self-Organizing Fuzzy-Neural-Network With Adaptive Computation Algorithm , 2014, IEEE Transactions on Cybernetics.

[52]  Chuen-Chien Lee,et al.  Fuzzy logic in control systems: fuzzy logic controller. I , 1990, IEEE Trans. Syst. Man Cybern..

[53]  Chung-Cheng Chen,et al.  Stability and Almost Disturbance Decoupling Analysis of Nonlinear System Subject to Feedback Linearization and Feedforward Neural Network Controller , 2008, IEEE Transactions on Neural Networks.

[54]  Shuzhi Sam Ge,et al.  Approximation-based adaptive tracking control of pure-feedback nonlinear systems with multiple unknown time-varying delays , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[55]  Feng Liu,et al.  A boundedness result for the direct heuristic dynamic programming , 2012, Neural Networks.

[56]  Haibo He,et al.  Adaptive Learning and Control for MIMO System Based on Adaptive Dynamic Programming , 2011, IEEE Transactions on Neural Networks.

[57]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[58]  Tao Li,et al.  Adaptive dynamic neuro-fuzzy system for traffic signal control , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[59]  D.Z. Fang,et al.  Adaptive fuzzy-logic SVC damping controller using strategy of oscillation energy descent , 2004, IEEE Transactions on Power Systems.

[60]  Jean-Jacques E. Slotine,et al.  Neural Network Control of Unknown Nonlinear Systems , 1989, 1989 American Control Conference.

[61]  Haibo He,et al.  Goal Representation Heuristic Dynamic Programming on Maze Navigation , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[62]  Derong Liu,et al.  Convergence analysis and application of fuzzy-HDP for nonlinear discrete-time HJB systems , 2015, Neurocomputing.

[63]  Yih-Guang Leu,et al.  Adaptive T-S fuzzy-neural modeling and control for general MIMO unknown nonaffine nonlinear systems using projection update laws , 2010, Autom..

[64]  P. B. Coaker,et al.  Applied Dynamic Programming , 1964 .

[65]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[66]  Yang Zhang,et al.  Design of Wide-Area Damping Controllers for Interarea Oscillations , 2008, IEEE Transactions on Power Systems.