Learning and tuning fuzzy logic controllers through reinforcements

A method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. It is shown that: the generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing.

[1]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[2]  Harley Bornbach,et al.  An introduction to mathematical learning theory , 1967 .

[3]  M. L. Tsetlin,et al.  Automaton theory and modeling of biological systems , 1973 .

[4]  Ebrahim H. Mamdani,et al.  A linguistic self-organizing process controller , 1979, Autom..

[5]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[7]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  Charles W. Anderson,et al.  Learning and problem-solving with multilayer connectionist systems (adaptive, strategy learning, neural networks, reinforcement learning) , 1986 .

[10]  C. Anderson.  Strategy Learning with Multilayer Connectionist Representations 1 , 1987 .

[11]  Charles W. Anderson,et al.  Strategy Learning with Multilayer Connectionist Representations , 1987 .

[12]  Y. Kasai,et al.  Electronically Controlled Continuously Variable Transmission (ECVT-II) , 1988, International Congress on Transportation Electronics,.

[13]  J.A. Bernard,et al.  Use of a rule-based system for process control , 1987, IEEE Control Systems Magazine.

[14]  Alberto L. Sangiovanni-Vincentelli,et al.  Efficient Parallel Learning Algorithms for Neural Networks , 1988, NIPS.

[15]  John A. Bernard,et al.  Use of a rule-based system for process control , 1988 .

[16]  Richard S. Sutton,et al.  Sequential Decision Problems and Neural Networks , 1989, NIPS 1989.

[17]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[18]  R. Sutton,et al.  Connectionist Learning for Control: An Overview , 1989 .

[19]  Yung-Yaw Chen,et al.  An Experiment-based Comparative Study of Fuzzy Logic Control , 1989, 1989 American Control Conference.

[20]  C.-C. Lee,et al.  An intelligent controller based on approximate reasoning and reinforcement learning , 1989, Proceedings. IEEE International Symposium on Intelligent Control 1989.

[21]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[22]  Jyh-Shing Roger Jang,et al.  A hierarchical approach to designing approximate reasoning-based controllers for dynamic physical systems , 1990, UAI.

[23]  Chuen-Chien Lee,et al.  A self‐learning rule‐based controller employing approximate reasoning and neural net concepts , 1991, Int. J. Intell. Syst..

[24]  Hamid R. Berenji.  An architecture for designing fuzzy logic controllers using neural networks , 1991 .

[25]  R. Yager.  An Alternative Procedure for the Calculation of Fuzzy Logic Controller Values(Journal of Japan Society for Fuzzy Theory and Systems) , 1991 .

[26]  Hamid R. Berenji,et al.  A reinforcement learning--based architecture for fuzzy logic control , 1992, Int. J. Approx. Reason..

[27]  H. R. Berenji,et al.  Fuzzy Logic Controllers , 1992 .

[28]  Jerry M. Mendel,et al.  Reinforcement-learning control and pattern recognition systems , 1994 .