A Study on Multi-Dimensional Fuzzy Q-learning for Intelligent Robots

Reinforcement learning is one of the most important learning methods for intelligent robots working in unknown/uncertain environments. Multi-dimensional fuzzy Q-learning, an extension of the Q-learning method, has been proposed in this study. The proposed method has been applied for an intelligent robot working in a dynamic environment. The rewards from the evaluation functions and the fuzzy Q-values generated by the neural networks (fuzzy Q-net) are expressed in vector forms in order to obtain optimal behaviors for several different purposes. By applying this learning method, evaluation and learning of fuzzy Q-values for the other behaviors can be carried out simultaneously in one trial. We express fuzzy states as the vector of fuzzy sets for input variables of the fuzzy Q-net. The behavior selection algorithm is also proposed in this study. The simulation results show the effectives of the proposed methods for a mobile robot selects optimal behavior in a dynamic environment.

[1]  James L. Crowley,et al.  Incremental supervised learning for mobile robot reactive control , 1997, Robotics Auton. Syst..

[2]  Spyros G. Tzafestas,et al.  Fuzzy reinforcement learning control for compliance tasks of robotic manipulators , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[3]  Tiziana D'Orazio,et al.  Neural Q-learning control architectures for a wall-following behavior , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[4]  Dong-Soo Kwon,et al.  Human-robot cooperation strategy for interactive robot soccer by fuzzy Q-learning , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[5]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[6]  N. H. C. Yung,et al.  A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Yoshikazu Arai,et al.  Collision avoidance in multi-robot systems based on multi-layered reinforcement learning , 1999, Robotics Auton. Syst..

[8]  Jacky Baltes,et al.  Path-Tracking Control of Non-Holonomic Car-Like Robot with Reinforcement Learning , 1999, New Zealand Computer Science Research Students' Conference.

[9]  Lionel Jouffe,et al.  Fuzzy inference system learning by reinforcement methods , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[10]  Minoru Asada,et al.  Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning , 2005, Machine Learning.

[11]  José del R. Millán,et al.  Rapid, safe, and incremental learning of navigation strategies , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Toshio Fukuda,et al.  Multi-dimensional reinforcement learning using a vector Q-net - Application to mobile robots , 2003 .

[13]  Dongbing Gu,et al.  Reinforcement learning of fuzzy logic controllers for quadruped walking robots , 2002 .

[14]  Chin-Teng Lin,et al.  A reinforcement neuro-fuzzy combiner for multiobjective control , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[15]  Antonio Pedro Aguiar,et al.  Regulation of a nonholonomic dynamic wheeled mobile robot with parametric modeling uncertainty using Lyapunov functions , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[16]  . A.Benyettou,et al.  Fuzzy Reinforcement Rectilinear Trajectory Learning , 2004 .

[17]  Luis Montano,et al.  A KINEMATIC AND DYNAMIC MODEL-BASED MOTION CONTROLLER FOR MOBILE ROBOTS , 2002 .

[18]  José del R. Millán,et al.  Learning efficient reactive behavioral sequences from basic reflexes in a goal-directed autonomous robot , 1994 .

[19]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[20]  Yau-Hwang Kuo,et al.  A parallel fuzzy inference model with distributed prediction scheme for reinforcement learning , 1998, IEEE Trans. Syst. Man Cybern. Part B.