Heading control for an Autonomous Underwater Vehicle using ELM-based Q-learning

Heading control is an important part of Autonomous Underwater Vehicle (AUV) control. But it's control performance is restricted to the uncertainty environments, and lack of understanding of dynamic characteristics of AUV. As a model-free method, the Q-learning achieves its control motivation by interacting with the environment and maximizing a reward, so suits the complicated applications in heading control of AUV. However, Q Learning algorithms are not competent for continuous space problems. So, Extreme Learning Machine(ELM) is proposed to guarantee the generalization performance and work with continuous states and actions. In this paper, the method of using ELM based Q Learning is proposed for heading control. The results have shown that the proposed method for heading control has good performance.

[1]  D. Cecchi,et al.  Autonomous underwater vehicles for scientific and naval operations , 2004 .

[2]  Alexander Zelinsky,et al.  Reinforcement Learning applied to the control of an Autonomous Underwater Vehicle , 1999 .

[3]  Monique Chyba,et al.  Autonomous underwater vehicles , 2009 .

[4]  Yves Chauvin,et al.  Backpropagation: theory, architectures, and applications , 1995 .

[5]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[6]  Joan Batlle,et al.  Hybrid coordination of reinforcement learning-based behaviors for AUV control , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[7]  Chris Gaskett,et al.  Q-Learning for Robot Control , 2002 .

[8]  Wettergreen Alexander Zelinsky.  Reinforcement Learning applied to the control of an AutonolDOlls ·U nclerwater .Vehicle , 1998 .

[9]  Ikuo Yamamoto.  Autonomous underwater vehicles and aviation robots , 2018 .

[10]  Wei Sun,et al.  Q Learning Based on Self-organizing Fuzzy Radial Basis Function Network , 2006, ISNN.

[11]  Andrew James Smith,et al.  Applications of the self-organising map to reinforcement learning , 2002, Neural Networks.

[12]  Longxin Lin,et al.  Supervised Neural Q_learning based Motion Control for Bionic Underwater Robots , 2010 .

[13]  Soroush Vahid,et al.  Modeling and Control of Autonomous Underwater Vehicle (AUV) In Heading and Depth Attitude via PPD Controller with State Feedback , 2016 .

[14]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[15]  Guang-Bin Huang,et al.  Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions , 1998, IEEE Trans. Neural Networks.

[16]  Nema Dean,et al.  Q-Learning: Flexible Learning About Useful Utilities , 2013, Statistics in Biosciences.

[17]  E. Feron,et al.  Robust hybrid control for autonomous vehicle motion planning , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[18]  Franz S. Hover,et al.  Modeling and control of an Odyssey III AUV through system identification tests , 2003 .

[19]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).