Efficient reinforcement learning through dynamic symbiotic evolution for TSK-type fuzzy controller design

In this paper, efficient reinforcement learning through dynamic-form symbiotic evolution (DSE) is proposed for solving nonlinear control problems. Compared with traditional symbiotic evolution, DSE uses the sequential search-based dynamic evolution (SSDE) method to generate an initial population and to determine dynamic mutation points. Therefore, better chromosomes will be initially generated while better mutation points will be determined for performing dynamic mutation. The proposed DSE design method was applied to different control systems, including the cart-pole balancing system and the water bath temperature control system, and control problems were simulated on these systems. The proposed DSE method was verified to be efficient and superior for solving these control problems and from comparisons with some traditional genetic algorithms.

[1]  Paul M. B. Vitányi,et al.  Theories of learning , 2007 .

[2]  Chin-Teng Lin,et al.  Reinforcement structure/parameter learning for neural-network-based fuzzy logic control systems , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[3]  Cheng-Jian Lin,et al.  A GA-based neural fuzzy system for temperature control , 2004, Fuzzy Sets Syst..

[4]  Xin Xu,et al.  Residual-gradient-based neural reinforcement learning for the optimal control of an acrobot , 2002, Proceedings of the IEEE Internatinal Symposium on Intelligent Control.

[5]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  Abdollah Homaifar,et al.  Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[7]  Richard S. Sutton,et al.  Landmark learning: An illustration of associative search , 1981, Biological Cybernetics.

[8]  Chin-Teng Lin,et al.  GA-based fuzzy reinforcement learning for control of a magnetic bearing system , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[9]  C. L. Karr,et al.  Fuzzy control of pH using genetic algorithms , 1993, IEEE Trans. Fuzzy Syst..

[10]  R. H. Cannon,et al.  Dynamics of Physical Systems , 1967 .

[11]  L. D. Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[12]  Ka Cheok,et al.  A ball balancing demonstration of optimal and disturbance-accomodating control , 1987, IEEE Control Systems Magazine.

[13]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 2004 .

[14]  Ka Cheok,et al.  A ball balancing demonstration of optimal and disturbance-accomodating control , 1987 .

[15]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[16]  Sigeru Omatu,et al.  Process control by on-line trained neural controllers , 1992, IEEE Trans. Ind. Electron..

[17]  Chin-Teng Lin,et al.  Genetic Reinforcement Learning through Symbiotic Evolution for Fuzzy Controller Design , 2022 .

[18]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[19]  Z. Deng,et al.  Competitive Takagi-Sugeno fuzzy reinforcement learning , 2001, Proceedings of the 2001 IEEE International Conference on Control Applications (CCA'01) (Cat. No.01CH37204).

[20]  Chin-Teng Lin,et al.  An online self-constructing neural fuzzy inference network and its applications , 1998, IEEE Trans. Fuzzy Syst..

[21]  C. S. George Lee,et al.  Neural fuzzy systems: a neuro-fuzzy synergism to intelligent systems , 1996 .

[22]  P. Kokotovic,et al.  Nonlinear control via approximate input-output linearization: the ball and beam example , 1992 .

[23]  Chia-Feng Juang,et al.  A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms , 2002, IEEE Trans. Fuzzy Syst..

[24]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[25]  Cheng Jian Lin,et al.  Nonlinear System Control Using Compensatory Neuro-Fuzzy Networks(Optimization and Control)( Nonlinear Theory and its Applications) , 2003 .

[26]  W. Estes.  Toward a Statistical Theory of Learning. , 1994 .

[27]  Ovidiu Grigore,et al.  Reinforcement learning neural network used in control of nonlinear systems , 2000, Proceedings of IEEE International Conference on Industrial Technology 2000 (IEEE Cat. No.00TH8482).

[28]  B. Pasik-Duncan,et al.  Adaptive Control , 1996, IEEE Control Systems.

[29]  John S. Edwards,et al.  The Hedonistic Neuron: A Theory of Memory, Learning and Intelligence , 1983 .

[30]  Charles W. Anderson,et al.  Learning and problem-solving with multilayer connectionist systems (adaptive, strategy learning, neural networks, reinforcement learning) , 1986 .