Mobile Robot Path Planning Based on Q-ANN

Path planning is a difficult part of the navigation task for the mobile robot under dynamic and unknown environment. It needs to solve a mapping relationship between the sensing space and the action space. The relationship can be achieved through different ways. But it is difficult to be expressed by an accurate equation. This paper uses multi-layer feed forward artificial neural network (ANN) to construct a path-planning controller by its powerful nonlinear functional approximation. Then the path planning task is simplified to a classified problem which are five state-action mapping relationship. One reinforcement learning method, Q-learning, is used to collect training samples for the ANN controller. At last the trained controller runs in the simulation environment and retrains itself furthermore combining the reinforcement signal during the interaction with the environment. Strategy based on the combination of ANN and Q-learning, Q-ANN, is better than using only one of the two methods. The simulation result also shows that the strategy can find the optimal path than using Q-learning only.

[1]  Philippe Lucidarme,et al.  An evolutionary algorithm for multi-robot unsupervised learning , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[2]  Il Hong Suh,et al.  A novel dynamic priority-based action-selection-mechanism integrating a reinforcement learning , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[3]  Meng Wang,et al.  Fuzzy logic based robot path planning in unknown environment , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[4]  Lu Jun.  Research on reinforcement learning and its application to mobile robot , 2004 .

[5]  Naoyuki Kubota,et al.  A spiking neural network for behavior learning of a mobile robot in a dynamic environment , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[6]  Min Feng.  Application of reinforcement learning based on artificial neural network to robot soccer , 2004 .

[7]  Min Guo,et al.  Reinforcement Learning Neural Network to the Problem of Autonomous Mobile Robot Obstacle Avoidance , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[8]  Yuan-Chun Li,et al.  Neuro/fuzzy behavior-based control of a mobile robot in unknown environments , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).