A fuzzy Q-learning approach to navigation of an autonomous robot

The proposed algorithm takes advantage of coupling fuzzy logic and Q-learning to fulfill requirements of autonomous navigations. Fuzzy if-then rules provide a reliable decision making framework to handle uncertainties, and also allow incorporation of heuristic knowledge. Dynamic structure of Q-learning makes it a promising tool to adjust fuzzy inference parameters when little or no prior knowledge is available about the world. To robot, the world is modeled into a set of state-action pairs. For each fuzzified state, there are some suggested actions. States are related to their corresponding actions via fuzzy if-then rules based on human reasoning. The robot selects the most encouraged action for each state through online experiences. Efficiency of the proposed method is validated through experiments on a simulated Khepera robot.

[1]  Pierre-Yves Glorennec,et al.  A Simple Goal Seeking Navigation Method for a Mobile Robot Using Human Sense, Fuzzy Logic and Reinforcement Learning , 2008, KES.

[2]  Francesco Mondada,et al.  Mobile Robot Miniaturisation: A Tool for Investigation in Control Algorithms , 1993, ISER.

[3]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[4]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[5]  Cipriano Galindo,et al.  Improving efficiency in mobile robot task planning through world abstraction , 2004, IEEE Transactions on Robotics.

[6]  Kunikazu Kobayashi,et al.  A Self-Organized Fuzzy-Neuro Reinforcement Learning System for Continuous State Space for Autonomous Robots , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[7]  Meng Joo Er,et al.  Online tuning of fuzzy inference systems using dynamic fuzzy Q-learning , 2004, IEEE Trans. Syst. Man Cybern. Part B.

[8]  Frank Klawonn,et al.  Foundations of fuzzy systems , 1994 .

[9]  Spyros G. Tzafestas,et al.  Motion control for mobile robot obstacle avoidance and navigation: a fuzzy logic-based approach , 2003 .

[10]  Lionel Jouffe,et al.  Fuzzy inference system learning by reinforcement methods , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[11]  Alessandro Saffiotti,et al.  The uses of fuzzy logic in autonomous robot navigation , 1997, Soft Comput..

[12]  Ronald C. Arkin,et al.  Motor schema based navigation for a mobile robot: An approach to programming by behavior , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[13]  Pierre-Yves Glorennec,et al.  A simple goal seeking navigation method for a mobile robot using human sense, fuzzy logic and reinforcement learning , 2008 .

[14]  Hichem Maaref,et al.  Fuzzy navigation strategy: application to two distinct autonomous mobile robots , 1997, Robotica.

[15]  George A. Bekey,et al.  A reinforcement-learning approach to reactive control policy design for autonomous robots , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[16]  Nouri Masmoudi,et al.  A Fuzzy Logic Based Navigation of a Mobile Robot , 2008 .

[17]  Pierre Yves Glorennec,et al.  Reinforcement Learning: an Overview , 2000 .

[18]  R. James Firby,et al.  An Investigation into Reactive Planning in Complex Domains , 1987, AAAI.

[19]  Bakir Lacevic,et al.  A 3-level autonomous mobile robot navigation system designed by using reasoning/search approaches , 2006, Robotics Auton. Syst..

[20]  H Maaref,et al.  Sensor-based fuzzy navigation of an autonomous mobile robot in an indoor environment , 2000 .

[21]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .