Qualitative models for adaptive critic neurocontrol

We demonstrate the use of qualitative models in the dual heuristic programming (DHP) method of training neurocontrollers. Two fuzzy approaches to developing qualitative models are explored: a priori application of problem specific knowledge, and estimation of a first order TSK fuzzy model. These approaches are demonstrated respectively on the cart-pole system and a nonlinear multiple-input-multiple-output plant proposed by Narendra. In both cases we find that a simplified model based on a Fuzzy framework enables better performance to be obtained as compared to use of non-fuzzy models of equivalent complexity. In both cases we use models that, while poor as one-step predictors, achieve effectiveness in the DHP training context equivalent to that of exact analytic models.

[1]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  Donald A. Sofge,et al.  Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches , 1992 .

[4]  Snehasis Mukhopadhyay,et al.  Adaptive control of nonlinear multivariable systems using neural networks , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[5]  Richard S. Sutton,et al.  A Menu of Designs for Reinforcement Learning Over Time , 1995 .

[6]  Roberto A. Santiago,et al.  Adaptive critic designs: A case study for neurocontrol , 1995, Neural Networks.

[7]  George G. Lendaris,et al.  More on training strategies for critic and action neural networks in dual heuristic programming method , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[8]  Nikita A. Visnevski.  Control of a nonlinear multivariable system with adaptive critic designs , 1997 .

[9]  Donald C. Wunsch,et al.  Adaptive critic designs and their applications , 1997 .

[10]  George G. Lendaris,et al.  Training strategies for critic and action neural networks in dual heuristic programming method , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[11]  J. Yen,et al.  Fuzzy Logic: Intelligence, Control, and Information , 1998 .

[12]  George G. Lendaris,et al.  DESIGNING (APPROXIMATE) OPTIMAL CONTROLLERS via DHP ADAPTIVE CRITICS & NEURAL NETWORKS , 1999 .

[13]  Thaddeus T. Shannon.  Partial, noisy and qualitative models for adaptive critic based neurocontrol , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[14]  George G. Lendaris,et al.  A comparison of training algorithms for DHP adaptive critic neurocontrol , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).