Hybrid Control for Robot Navigation - A Hierarchical Q-Learning Algorithm

Autonomous mobile robots have been widely studied and applied not only as a test bed to academically demonstrate the achievement of artificial intelligence but also as an essential component of industrial and home automation. Mobile robots have many potential applications in routine or dangerous tasks such as delivery of supplies in hospitals, cleaning of offices, and operations in a nuclear plant. One of the fundamental and critical research areas in mobile robotics is navigation, which generally includes local navigation and global navigation. Local navigation, often called reactive control, learns or plans the local paths using the current sensory inputs without prior complete knowledge of the environment. Global navigation, often called deliberate control, learns or plans the global paths based on a relatively abstract and complete knowledge about the environment. In this article, hybrid control architecture is conceived via combining reactive and deliberate control using a hierarchical Q-learning (HQL) algorithm.

[1]  Yoram Koren,et al.  Real-time obstacle avoidance for fact mobile robots , 1989, IEEE Trans. Syst. Man Cybern..

[2]  Se-Young Oh,et al.  Hybrid Control for Autonomous Mobile Robot Navigation Using Neural Network Based Behavior Modules and Environment Classification , 2003, Auton. Robots.

[3]  Zonghai Chen,et al.  Grey Reinforcement Learning for Incomplete Information Processing , 2006, TAMC.

[4]  Frédéric Lerasle,et al.  Topological navigation and qualitative localization for indoor environment using multi-sensory perception , 2002, Robotics Auton. Syst..

[5]  Dídac Busquets,et al.  A Multiagent Approach to Qualitative Landmark-Based Navigation , 2003, Auton. Robots.

[6]  Luis Moreno,et al.  Navigation of mobile robots: open questions , 2000, Robotica.

[7]  Sebastian Thrun,et al.  Learning Metric-Topological Maps for Indoor Mobile Robot Navigation , 1998, Artif. Intell..

[8]  Benjamin Kuipers,et al.  Qualitative and Quantitative Simulation: Bridging the Gap , 1997, Artif. Intell..

[9]  Arturo de la Escalera,et al.  Using learned visual landmarks for intelligent topological navigation of mobile robots , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[10]  Toshiyuki Kondo,et al.  A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control , 2003, Robotics Auton. Syst..

[11]  Viii Supervisor.  Sonar-Based Real-World Mapping and Navigation , 2001 .

[12]  Chung-Chun Kung,et al.  Affine Takagi-Sugeno fuzzy modelling algorithm by fuzzy c-regression models clustering with a novel cluster validity criterion , 2007 .

[13]  Sebastian Thrun,et al.  Probabilistic Algorithms in Robotics , 2000, AI Mag..

[14]  Hugh F. Durrant-Whyte,et al.  An evidential approach to map-building for autonomous vehicles , 1998, IEEE Trans. Robotics Autom..

[15]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[16]  Simon X. Yang,et al.  A Neuro-fuzzy Controller for Reactive Navigation of a Behaviour-Based Mobile Robot , 2005, ISNN.

[17]  Benjamin Kuipers,et al.  Cognitive Maps for Planetary Rovers , 2001, Auton. Robots.

[18]  Jürgen Schmidhuber,et al.  HQ-Learning , 1997, Adapt. Behav..

[19]  Arturo de la Escalera,et al.  Learning Visual Landmarks for Mobile Robot Topological Navigation , 2005, Machine Learning and Robot Perception.

[20]  Sebastian Thrun,et al.  A Probabilistic On-Line Mapping Algorithm for Teams of Mobile Robots , 2001, Int. J. Robotics Res..

[21]  Stuart J. Russell,et al.  Reinforcement Learning with Hierarchies of Machines , 1997, NIPS.

[22]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[23]  John Hallam,et al.  Behaviour selection on a mobile robot using W-learning , 2002 .

[24]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[25]  Benjamin Kuipers,et al.  Towards a general theory of topological maps , 2004, Artif. Intell..

[26]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Nikitas M. Sgouros.  Qualitative Navigation for Autonomous Wheelchair Robots in Indoor Environments , 2002, Auton. Robots.

[29]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[30]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..