ABSTRACT This paper presents the application of the reinforcement learning algorithms to the task of autonomous determination of the ship trajectory during the in-harbour and harbour approaching manoeuvres. Authors used Markov decision processes formalism to build up the background of algorithm presentation. Two versions of RL algorithms were tested in the simulations: discrete (Q-learning) and continuous form (Least-Squares Policy Iteration). The results show that in both cases ship trajectory can be found. However discrete Q-learning algorithm suffered from many limitations (mainly curse of dimensionality) and practically is not applicable to the examined task. On the other hand, LSPI gave promising results. To be fully operational, proposed solution should be extended by taking into account ship heading and velocity and coupling with advanced multi-variable controller.
[1]
Bart De Schutter,et al.
Reinforcement Learning and Dynamic Programming Using Function Approximators
,
2010
.
[2]
Guo Chen,et al.
Reinforcement learning control for ship steering based on general fuzzified CMAC
,
2004,
2004 5th Asian Control Conference (IEEE Cat. No.04EX904).
[3]
Michail G. Lagoudakis,et al.
Least-Squares Policy Iteration
,
2003,
J. Mach. Learn. Res..
[4]
Peter Dayan,et al.
Q-learning
,
1992,
Machine Learning.
[5]
Andrzej Rak,et al.
Maneuvering control and trajectory tracking of very large crude carrier
,
2007
.
[6]
Richard S. Sutton,et al.
Reinforcement Learning: An Introduction
,
1998,
IEEE Trans. Neural Networks.