Model predictive ship collision avoidance based on Q-learning beetle swarm antenna search and neural networks

Abstract Real-time collision avoidance with full consideration of ship maneuverability, collision risks and International Regulations for Preventing Collisions at Sea (COLREGs) is difficult in multi-ship encounters. To deal with this problem, a novel method is proposed based on model predictive control (MPC), an improved Q-learning beetle swarm antenna search (I-Q-BSAS) algorithm and neural networks. The main idea of this method is to use a neural network to approximate an inverse model based on decisions made with MPC for collision avoidance. Firstly, the predictive collision avoidance strategy is established following the MPC concept incorporating an I-Q-BSAS algorithm to solve the optimization problem. Meanwhile, the relative collision motion states in typical encounters are collected for training an inverse neural network model, which is used as an approximated optimal policy of MPC. Moreover, to deal with uncertain dynamics, the obtained policy is reinforced by long-term retraining based on an aggregation of on-policy and off-policy data. Ship collision avoidance in multi-ship encounters can be achieved by weighting the outputs of the neural network model with respect to different target ships. Simulation experiments under several typical and multi-ship encounters are carried out using the KVLCC2 ship model to verify the effectiveness of the proposed method.