A REINFORCEMENT LEARNING ALGORITHM WITH EVOLVING FUZZY NEURAL NETWORKS

Abstract The synergy of the two paradigms, neural network and fuzzy inference system, has given rise to rapidly emerging filed, neuro-fuzzy systems. Evolving neuro-fuzzy systems are intended to use online learning to extract knowledge from data and perform a high-level adaptation of the network structure. We explore the potential of evolving neuro-fuzzy systems in reinforcement learning (RL) applications. In this paper, a novel on-line sequential learning evolving neuro-fuzzy model design for RL is proposed. We develop a dynamic evolving fuzzy neural network (DENFIS) function approximation approach to RL systems. Potential of this approach is demonstrated through a case study–-two-link robot manipulator. Simulation results have demonstrated that the proposed approach performs well in reinforcement learning problems.

[1]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[2]  Lionel Jouffe,et al.  Fuzzy inference system learning by reinforcement methods , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[3]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[4]  Derek A. Linkens,et al.  Learning systems in intelligent control: an appraisal of fuzzy, neural and genetic algorithm control applications , 1996 .

[5]  Michael J. Watts,et al.  A Decade of Kasabov's Evolving Connectionist Systems: A Review , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[6]  José M. N. Vieira,et al.  Neuro-Fuzzy Systems: A Survey , 2004 .

[7]  Bohdana Ratitch On characteristics of markov decision processes and reinforcement learning in large domains , 2005 .

[8]  Kurt Hornik,et al.  FEED FORWARD NETWORKS ARE UNIVERSAL APPROXIMATORS , 1989 .

[9]  Nikola K. Kasabov,et al.  Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[10]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[11]  R.J. Williams,et al.  Reinforcement learning is direct adaptive optimal control , 1991, IEEE Control Systems.

[12]  Meng Joo Er,et al.  Online tuning of fuzzy inference systems using dynamic fuzzy Q-learning , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[13]  김재현,et al.  Fuzzy-Q learning , 1996 .

[14]  Nikola Kasabov,et al.  Foundations Of Neural Networks, Fuzzy Systems, And Knowledge Engineering [Books in Brief] , 1996, IEEE Transactions on Neural Networks.

[15]  Andrew W. Moore,et al.  Generalization in Reinforcement Learning: Safely Approximating the Value Function , 1994, NIPS.