Reinforcement learning for an ART-based fuzzy adaptive learning control network

This paper proposes a reinforcement fuzzy adaptive learning control network (RFALCON), constructed by integrating two fuzzy adaptive learning control networks (FALCON), each of which has a feedforward multilayer network and is developed for the realization of a fuzzy controller. One FALCON performs as a critic network (fuzzy predictor), the other as an action network (fuzzy controller). Using temporal difference prediction, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network performs a stochastic exploratory algorithm to adapt itself according to the internal reinforcement signal. An ART-based reinforcement structure/parameter-learning algorithm is developed for constructing the RFALCON dynamically. During the learning process, structure and parameter learning are performed simultaneously. RFALCON can construct a fuzzy control system through a reward/penalty signal. It has two important features; it reduces the combinatorial demands of system adaptive linearization, and it is highly autonomous.

[1]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[2]  W. Rudin.  Principles of mathematical analysis , 1964 .

[3]  Thomas M. Cover,et al.  The two-armed-bandit problem with time-invariant finite memory , 1970, IEEE Trans. Inf. Theory.

[4]  A. Dickinson.  Contemporary Animal Learning Theory , 1981 .

[5]  John S. Edwards,et al.  The Hedonistic Neuron: A Theory of Memory, Learning and Intelligence , 1983 .

[6]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[8]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Charles W. Anderson,et al.  Strategy Learning with Multilayer Connectionist Representations , 1987 .

[10]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[11]  J. Franklin.  Input space representation for refinement learning control , 1989, Proceedings. IEEE International Symposium on Intelligent Control 1989.

[12]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[13]  C.-C. Lee,et al.  An intelligent controller based on approximate reasoning and reinforcement learning , 1989, Proceedings. IEEE International Symposium on Intelligent Control 1989.

[14]  Chuen-Chien Lee.  FUZZY LOGIC CONTROL SYSTEMS: FUZZY LOGIC CONTROLLER - PART I , 1990 .

[15]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[16]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[17]  Chin-Teng Lin,et al.  Neural-Network-Based Fuzzy Logic Control and Decision System , 1991, IEEE Trans. Computers.

[18]  Oliver G. Selfridge,et al.  Real-time learning: a ball on a beam , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[19]  Hamid R. Berenji,et al.  Learning and tuning fuzzy logic controllers through reinforcements , 1992, IEEE Trans. Neural Networks.

[20]  P. Kokotovic,et al.  Nonlinear control via approximate input-output linearization: the ball and beam example , 1992 .

[21]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[22]  Chin-Teng Lin,et al.  Real-time supervised structure/parameter learning for fuzzy neural network , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[23]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[24]  Chin-Teng Lin,et al.  Reinforcement structure/parameter learning for neural-network-based fuzzy logic control systems , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[25]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks - Part 2: Clustering , 1993, IEEE Trans. Fuzzy Syst..

[26]  W. Estes.  Toward a Statistical Theory of Learning. , 1994 .

[27]  V. Gullapalli,et al.  Acquiring robot skills via reinforcement learning , 1994, IEEE Control Systems.

[28]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[29]  Richard S. Sutton,et al.  A Menu of Designs for Reinforcement Learning Over Time , 1995 .

[30]  Cheng-Jian Lin,et al.  Fuzzy adaptive learning control network with on-line neural learning , 1995 .

[31]  Shigeo Abe,et al.  Neural Networks and Fuzzy Systems , 1996, Springer US.

[32]  Chin-Teng Lin,et al.  An ART-based fuzzy adaptive learning control network , 1997, IEEE Trans. Fuzzy Syst..