Toward Enhanced Support for Ship Sailing

Ship sailing is a complex endeavour, requiring carefully considered proactive and reactive strategies in choosing the course of action that best suits the various events to be managed. Humans are already supported by different technologies for sailing, however these technologies are usually available in isolation. In this paper we show how to use simultaneously three different technologies by fusing their information in order to provide enhanced support for ship sailing. To the best of our knowledge no similar approach is reported in the literature from an operational point of view. In particular, we show how to fuse the video acquired from a camera with the information available from a radar/Lidar and an AIS receiver. The video frames are analyzed in order to automatically detect surrounding ships and seamarks, the Lidar is used to determine the average or minimum distance from the ship to the acquired targets and finally the AIS receiver logs are queried to determine, if available, useful information related to the surrounding ships, such as their geographic location, type of ship etc. Our experimental results are promising and encouraging. We believe that the simultaneous use of these technologies is a step towards fully autonomous ship sailing.

[1]  Jun Yu Li,et al.  Global and Local Context-Aware Ship Detector for High-Resolution SAR Images , 2023, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Zenghui Zhang,et al.  A domain specific knowledge extraction transformer method for multisource satellite-borne SAR images ship detection , 2023, ISPRS Journal of Photogrammetry and Remote Sensing.

[3]  Hui Li,et al.  Silicon energy bulk material cargo ship detection and tracking method combining YOLOv5 and DeepSort , 2023, Energy Reports.

[4]  T. Laroussi,et al.  Approximate MLE based automatic bilateral censoring CFAR ship detection for complex scenes of log-normal sea clutter in SAR imagery , 2023, Digit. Signal Process..

[5]  J. Pei,et al.  Nighttime Infrared Ship Target Detection Based on Two-channel Image Separation Combined with Saliency Mapping of Local Grayscale Dynamic Range , 2022, Infrared Physics & Technology.

[6]  D. Abel,et al.  Maritime Environment Perception Based on Deep Learning , 2022, IEEE Transactions on Intelligent Transportation Systems.

[7]  Nour Elhouda Chaffa,et al.  The combination of singular values decomposition with constant false alarm algorithms to enhance ship detection in a polarimetric SAR application , 2022, Remote Sensing Applications: Society and Environment.

[8]  Xin Lou,et al.  Generative knowledge transfer for ship detection in SAR images , 2022, Comput. Electr. Eng..

[9]  Parneet Kaur,et al.  Sea Situational Awareness (SeaSAw) Dataset , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  R. Mansour,et al.  Intelligent deep learning-enabled autonomous small ship detection and classification model , 2022, Comput. Electr. Eng..

[11]  Kexin Bao,et al.  Ship Detection Based on LiDAR and Visual Information Fusion , 2022, 2022 Conference on Lasers and Electro-Optics (CLEO).

[12]  Bing Han,et al.  Accurate Real-time Ship Target detection Using Yolov4 , 2021, 2021 6th International Conference on Transportation Information and Safety (ICTIS).

[13]  Zhuo Chen,et al.  Fast Ship Detection in Remote Sensing Images Based on Multi-Attention Mechanism , 2021, ICACS.

[14]  Peng Liu,et al.  Research on Ship Detection in the SAR Image Algorithm Based on Improved SSD , 2021, AIPR.

[15]  Yuxu Lu,et al.  An enhanced CNN-enabled learning method for promoting ship detection in maritime surveillance system , 2021 .

[16]  Haoyu Yin,et al.  Ship Detection in Large-scale SAR Images Based on Dense Spatial Attention and Multi-level Feature Fusion , 2021, ACM TUR-C.

[17]  Z. Shao,et al.  GLSD: The Global Large-Scale Ship Database and Baseline Evaluations , 2021, ArXiv.

[18]  Benzheng Shi,et al.  Improved Detection and Recognition of Sea Surface Ships Based on YOLOv3 , 2021, ICECC.

[19]  Jianlong Fu,et al.  Learning Spatio-Temporal Transformer for Visual Tracking , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[20]  Lin Li,et al.  Infrared ship detection based on time fluctuation feature and space structure feature in sun-glint scene , 2021 .

[21]  Farhan Mumtaz,et al.  Ship Detection and Tracking in Inland Waterways Using Improved YOLOv3 and Deep SORT , 2021, Symmetry.

[22]  Johan Lilius,et al.  ABOShips - An Inshore and Offshore Maritime Vessel Detection Dataset with Precise Annotations , 2021, Remote. Sens..

[23]  Y. Coady,et al.  Ship Detection in Satellite Optical Imagery , 2020, AICCC.

[24]  Long Han,et al.  Asymmetric Convolution-Based Neural Network for SAR Ship Detection from Scratch , 2020, ICCPR.

[25]  Bo Li,et al.  Ship detection and classification from optical remote sensing images: A survey , 2020 .

[26]  Yuxing Mao,et al.  An Anchor-free SAR Ship Detector with Only 1.17M Parameters , 2020 .

[27]  Yuxing Mao,et al.  Network Slimming Method for SAR Ship Detection Based on Knowlegde Distillation , 2020 .

[28]  Chaozhong Wu,et al.  Deep learning for autonomous ship-oriented small ship detection , 2020 .

[29]  Simo Särkkä,et al.  Sensors and AI Techniques for Situational Awareness in Autonomous Ships: A Review , 2020, IEEE Transactions on Intelligent Transportation Systems.

[30]  Baojun Qiao,et al.  Phase spectrum based automatic ship detection in synthetic aperture radar images , 2020 .

[31]  Xinghai Yang,et al.  Ship Target Detection Algorithm Based on Improved YOLOv3 , 2020, ICBDT.

[32]  Xi Zhao,et al.  Asymmetric and Square Convolutional Neural Network for SAR Ship Detection from Scratch , 2020, ICBIP.

[33]  Shun Zhang,et al.  Mcships: A Large-Scale Ship Dataset For Detection And Fine-Grained Categorization In The Wild , 2020, 2020 IEEE International Conference on Multimedia and Expo (ICME).

[34]  Trevor Darrell,et al.  Quasi-Dense Similarity Learning for Multiple Object Tracking , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Lin He,et al.  Ship Detection Method Based on Gabor Filter and Fast RCNN Model in Satellite Images of Sea , 2019, CSAE.

[36]  Antonio Moccia,et al.  Integration of Automatic Identification System (AIS) Data and Single-Channel Synthetic Aperture Radar (SAR) Images by SAR-Based Ship Velocity Estimation for Maritime Situational Awareness , 2019, Remote. Sens..

[37]  Xiongwei Wu,et al.  Recent Advances in Deep Learning for Object Detection , 2019, Neurocomputing.

[38]  Bo Fu,et al.  Multi-Sensor Integrated Navigation System for Ships Based on Adaptive Kalman Filter , 2019, 2019 IEEE International Conference on Mechatronics and Automation (ICMA).

[39]  Carlos López-Martínez,et al.  Large-Scale Automatic Vessel Monitoring Based on Dual-Polarization Sentinel-1 and AIS Data , 2019, Remote. Sens..

[40]  Wei Wu,et al.  SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Junghoon Seo,et al.  RBox-CNN: rotated bounding box based CNN for ship detection in remote sensing image , 2018, SIGSPATIAL/GIS.

[42]  Jukka Heikkonen,et al.  An Efficient Multi-sensor Fusion Approach for Object Detection in Maritime Environments , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[43]  Jing Liu,et al.  A fusion algorithm of target dynamic information for asynchronous multi-sensors , 2018 .

[44]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[45]  Zhongyuan Wang,et al.  SeaShips: A Large-Scale Precisely Annotated Dataset for Ship Detection , 2018, IEEE Transactions on Multimedia.

[46]  Jun Zhang,et al.  Robust single stage detector based on two-stage regression for SAR ship detection , 2018, ICIAI '18.

[47]  Jia Liu,et al.  A New Ship Target Detection Algorithm Based on SVM in High Resolution SAR Images , 2017, ICAIP.

[48]  Dong Xu,et al.  Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition , 2017, ACM Comput. Surv..

[49]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[50]  Erhan Gundogdu,et al.  MARVEL: A Large-Scale Image Dataset for Maritime Vessels , 2016, ACCV.

[51]  Deepu Rajan,et al.  Video Processing From Electro-Optical Sensors for Object Detection and Tracking in a Maritime Environment: A Survey , 2016, IEEE Transactions on Intelligent Transportation Systems.

[52]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Jean-Yves Tourneret,et al.  Ship detection using SAR and AIS raw data for maritime surveillance , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[54]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[55]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[56]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[58]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Michael T. Wolf,et al.  VAIS: A dataset for recognizing maritime imagery in the visible and infrared spectrums , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[61]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[63]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[64]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[66]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[68]  Wu-Chih Hu,et al.  Robust real-time ship detection and tracking for visual surveillance of cage aquaculture , 2011, J. Vis. Commun. Image Represent..

[69]  Sheng-Zheng Wang,et al.  A multi-agent information fusion model for ship collision avoidance , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[70]  A Tiano,et al.  Multisensor Data Fusion Applied to Marine Integrated Navigation Systems , 2005 .

[71]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[72]  M. Lings,et al.  Articles , 1967, Soil Science Society of America Journal.

[73]  D. Xiang,et al.  Unsupervised Ship Detection in SAR Images Using Superpixels and CSPNet , 2023, IEEE Geoscience and Remote Sensing Letters.

[74]  Gang Li,et al.  Frequency-Adaptive Learning for SAR Ship Detection in Clutter Scenes , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[75]  Ruixiang Zhang,et al.  Fine-Grained Ship Detection in High-Resolution Satellite Images With Shape-Aware Feature Learning , 2023, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[76]  Norma Latif Fitriyani,et al.  CRAS-YOLO: A Novel Multi-Category Vessel Detection and Classification Model Based on YOLOv5s Algorithm , 2023, IEEE Access.

[77]  Zenghui Zhang,et al.  Information Reconstruction-Based Polarimetric Covariance Matrix for PolSAR Ship Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[78]  Yang Yang,et al.  Physics Guided Remote Sensing Image Synthesis Network for Ship Detection , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[79]  Zhen Ye,et al.  Feature Enhancement Pyramid and Shallow Feature Reconstruction Network for SAR Ship Detection , 2023, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[80]  L. Du,et al.  Semisupervised SAR Ship Detection Network via Scene Characteristic Learning , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[81]  M. Xing,et al.  Robust Multi-Ship Tracker in SAR Imagery by Fusing Feature Matching and Modified KCF , 2023, IEEE Geoscience and Remote Sensing Letters.

[82]  Cheng Chi,et al.  A Survey on Deep-Learning-Based Real-Time SAR Ship Detection , 2023, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[83]  Tong Zhang,et al.  Contour Modeling Arbitrary-Oriented Ship Detection From Very High-Resolution Optical Remote Sensing Imagery , 2023, IEEE Geoscience and Remote Sensing Letters.

[84]  Xiao-Ping Zhang,et al.  Proposal-Copula-Based Fusion of Spaceborne and Airborne SAR Images for Ship Target Detection⁎⁎ , 2022, Inf. Fusion.

[85]  Myung-Il Roh,et al.  Detection and tracking for the awareness of surroundings of a ship based on deep learning , 2021, J. Comput. Des. Eng..

[86]  R. Glenn Wright,et al.  Intelligent Autonomous Ship Navigation using Multi-Sensor Modalities , 2019, TransNav, the International Journal on Marine Navigation and Safety of Sea Transportation.

[87]  Peter H. N. de With,et al.  Ship Detection in Harbour Surveillance based on Large-Scale Data and CNNs , 2018, VISIGRAPP.

[88]  Jitendra Malik,et al.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  Huanxin Zou,et al.  Ship Surveillance by Integration of Space-borne SAR and AIS – Review of Current Research , 2014 .