Instance segmentation ship detection based on improved Yolov7 using complex background SAR images

It is significant for port ship scheduling and traffic management to be able to obtain more precise location and shape information from ship instance segmentation in SAR pictures. Instance segmentation is more challenging than object identification and semantic segmentation in high-resolution RS images. Predicting class labels and pixel-wise instance masks is the goal of this technique, which is used to locate instances in images. Despite this, there are now just a few methods available for instance segmentation in high-resolution RS data, where a remote-sensing image’s complex background makes the task more difficult. This research proposes a unique method for YOLOv7 to improve HR-RS image segmentation one-stage detection. First, we redesigned the structure of the one-stage fast detection network to adapt to the task of ship target segmentation and effectively improve the efficiency of instance segmentation. Secondly, we improve the backbone network structure by adding two feature optimization modules, so that the network can learn more features and have stronger robustness. In addition, we further modify the network feature fusion structure, improve the module acceptance domain to increase the prediction ability of multi-scale targets, and effectively reduce the amount of model calculation. Finally, we carried out extensive validation experiments on the sample segmentation datasets HRSID and SSDD. The experimental comparisons and analyses on the HRSID and SSDD datasets show that our model enhances the predicted instance mask accuracy, enhancing the instance segmentation efficiency of HR-RS images, and encouraging further enhancements in the projected instance mask accuracy. The suggested model is a more precise and efficient segmentation in HR-RS imaging as compared to existing approaches.

[1]  Guofu Yin,et al.  Deep Feature Interaction Network for Point Cloud Registration, With Applications to Optical Measurement of Blade Profiles , 2023, IEEE Transactions on Industrial Informatics.

[2]  Hao Sheng,et al.  Hybrid Motion Model for Multiple Object Tracking in Mobile Devices , 2023, IEEE Internet of Things Journal.

[3]  Wantao Liu,et al.  Lightweight algorithm for multi-scale ship detection based on high-resolution SAR images , 2023, International Journal of Remote Sensing.

[4]  Kinh Bac Dang,et al.  Multi-scale ship target detection using SAR images based on improved Yolov5 , 2023, Frontiers in Marine Science.

[5]  Tianwen Zhang,et al.  Synthetic Aperture Radar (SAR) Meets Deep Learning , 2023, Remote. Sens..

[6]  Xiangguang Leng,et al.  A Lightweight Model for Ship Detection and Recognition in Complex-Scene SAR Images , 2022, Remote. Sens..

[7]  Shunjun Wei,et al.  A Group-Wise Feature Enhancement-and-Fusion Network with Dual-Polarization Feature Enrichment for SAR Ship Detection , 2022, Remote. Sens..

[8]  Zhe Zeng,et al.  Ship detection based on deep learning using SAR imagery: a systematic literature review , 2022, Soft Computing.

[9]  Tianwen Zhang,et al.  RBFA-Net: A Rotated Balanced Feature-Aligned Network for Rotated SAR Ship Detection and Classification , 2022, Remote. Sens..

[10]  Tianwen Zhang,et al.  A Mask Attention Interaction and Scale Enhancement Network for SAR Ship Instance Segmentation , 2022, IEEE Geoscience and Remote Sensing Letters.

[11]  Tianwen Zhang,et al.  Shadow-Background-Noise 3D Spatial Decomposition Using Sparse Low-Rank Gaussian Properties for Video-SAR Moving Target Shadow Enhancement , 2022, IEEE Geoscience and Remote Sensing Letters.

[12]  Hai Wang,et al.  Ship Detection in SAR Images Based on Feature Enhancement Swin Transformer and Adjacent Feature Fusion , 2022, Remote. Sens..

[13]  Tianwen Zhang,et al.  HTC+ for SAR Ship Instance Segmentation , 2022, Remote. Sens..

[14]  Xiangguang Leng,et al.  An Improved Oriented Ship Detection Method in High-resolution SAR Image Based on YOLOv5 , 2022, Progress in Electromagnetics Research Symposium.

[15]  Zhibo Wan,et al.  Ore Image Classification Based on Improved CNN , 2022, Comput. Electr. Eng..

[16]  Chengjie Zong,et al.  An improved 3D point cloud instance segmentation method for overhead catenary height detection , 2022, Comput. Electr. Eng..

[17]  Jiaguo Li,et al.  Multi-Scale Ship Detection Algorithm Based on a Lightweight Neural Network for Spaceborne SAR Images , 2022, Remote. Sens..

[18]  X. Xing,et al.  SII-Net: Spatial Information Integration Network for Small Target Detection in SAR Images , 2022, Remote. Sens..

[19]  Zhibo Wan,et al.  CONTAINER SHIP CELL GUIDE ACCURACY CHECK TECHNOLOGY BASED ON IMPROVED 3D POINT CLOUD INSTANCE SEGMENTATION , 2022, Brodogradnja.

[20]  Shunjun Wei,et al.  Balance learning for ship detection from synthetic aperture radar remote sensing imagery , 2021, ISPRS Journal of Photogrammetry and Remote Sensing.

[21]  Zhejun Feng,et al.  An Improved Swin Transformer-Based Model for Remote Sensing Object Detection and Instance Segmentation , 2021, Remote. Sens..

[22]  Tianwen Zhang,et al.  Integrate Traditional Hand-Crafted Features into Modern CNN-based Models to Further Improve SAR Ship Classification Accuracy , 2021, 2021 7th Asia-Pacific Conference on Synthetic Aperture Radar (APSAR).

[23]  Boli Xiong,et al.  BiFA-YOLO: A Novel YOLO-Based Method for Arbitrary-Oriented Ship Detection in High-Resolution SAR Images , 2021, Remote. Sens..

[24]  Xiaoling Zhang,et al.  A polarization fusion network with geometric feature embedding for SAR ship classification , 2021, Pattern Recognit..

[25]  Zelin Zhang,et al.  Efficient image segmentation based on deep learning for mineral image classification , 2021, Advanced Powder Technology.

[26]  Jenq-Neng Hwang,et al.  GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation , 2021, IEEE Transactions on Image Processing.

[27]  Si-Wei Chen,et al.  Speckle-Free SAR Image Ship Detection , 2021, IEEE Transactions on Image Processing.

[28]  Wujie Zhou,et al.  Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[29]  Shanxi Li,et al.  SCCGAN: Style and Characters Inpainting Based on CGAN , 2021, Mobile Networks and Applications.

[30]  Chao Wang,et al.  MW-ACGAN: Generating Multiscale High-Resolution SAR Images for Ship Detection , 2020, Sensors.

[31]  Xiaoling Zhang,et al.  HyperLi-Net: A hyper-light deep learning network for high-accurate and high-speed ship detection from synthetic aperture radar imagery , 2020 .

[32]  Lorenzo Bruzzone,et al.  Multi-Scale Context Aggregation for Semantic Segmentation of Remote Sensing Images , 2020, Remote. Sens..

[33]  Licheng Jiao,et al.  Object Detection in High-Resolution Panchromatic Images Using Deep Models and Spatial Template Matching , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Bingliang Hu,et al.  Attention Mask R-CNN for Ship Detection and Segmentation From Remote Sensing Images , 2020, IEEE Access.

[35]  Chunhua Shen,et al.  BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Chen Wang,et al.  Precise and Robust Ship Detection for High-Resolution SAR Imagery Based on HR-SDNet , 2020, Remote. Sens..

[37]  Gong Cheng,et al.  Object Detection in Remote Sensing Images Based on Improved Bounding Box Regression and Multi-Level Features Fusion , 2020, Remote. Sens..

[38]  Yuning Jiang,et al.  SOLO: Segmenting Objects by Locations , 2019, ECCV.

[39]  Xueru Bai,et al.  Ship Detection Using Deep Convolutional Neural Networks for PolSAR Images , 2019, Remote. Sens..

[40]  Jun-Wei Hsieh,et al.  CSPNet: A New Backbone that can Enhance Learning Capability of CNN , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Fei Gao,et al.  Enhanced Feature Extraction for Ship Detection from Multi-Resolution and Multi-Scene Synthetic Aperture Radar (SAR) Images , 2019, Remote. Sens..

[42]  Qi Li,et al.  Dense Attention Pyramid Networks for Multi-Scale Ship Detection in SAR Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Xiaoling Zhang,et al.  Depthwise Separable Convolution Neural Network for High-Speed SAR Ship Detection , 2019, Remote. Sens..

[44]  Ping Luo,et al.  PolarMask: Single Shot Instance Segmentation With Polar Representation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Jonathan Li,et al.  Ship Detection Using a Fully Convolutional Network with Compact Polarimetric SAR Images , 2019, Remote. Sens..

[46]  Chen Wang,et al.  Object Detection and Instance Segmentation in Remote Sensing Imagery Based on Precise Mask R-CNN , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[47]  Nuno Vasconcelos,et al.  Cascade R-CNN: High Quality Object Detection and Instance Segmentation , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Xiaoling Zhang,et al.  High-Speed Ship Detection in SAR Images Based on a Grid Convolutional Neural Network , 2019, Remote. Sens..

[49]  Ling Shao,et al.  iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images , 2019, CVPR Workshops.

[50]  Yong Jae Lee,et al.  YOLACT: Real-Time Instance Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[51]  Lena Chang,et al.  Ship Detection Based on YOLOv2 for SAR Imagery , 2019, Remote. Sens..

[52]  Weiwei Sun,et al.  R-CNN-Based Ship Detection from High Resolution Remote Sensing Imagery , 2019, Remote. Sens..

[53]  Hong Zhang,et al.  Automatic Ship Detection Based on RetinaNet Using Multi-Resolution Gaofen-3 Imagery , 2019, Remote. Sens..

[54]  Kai Chen,et al.  Hybrid Task Cascade for Instance Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Weiwei Jiang,et al.  Simultaneous Ship Detection and Orientation Estimation in SAR Images Based on Attention Module and Angle Regression , 2018, Sensors.

[56]  Xiao Xiang Zhu,et al.  Vehicle Instance Segmentation From Aerial Image and Video Using a Multitask Learning Residual Fully Convolutional Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[58]  Jianwei Li,et al.  Ship detection in SAR images based on an improved faster R-CNN , 2017, 2017 SAR in Big Data Era: Models, Methods and Applications (BIGSARDATA).

[59]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[60]  Zhao Lin,et al.  A modified faster R-CNN based on CFAR algorithm for SAR ship detection , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[61]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[62]  Serge J. Belongie,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Jian Sun,et al.  Instance-Aware Semantic Segmentation via Multi-task Network Cascades , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Salvatore Calcagno,et al.  Fuzzy geometrical approach based on unit hyper-cubes for image contrast enhancement , 2015, 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[65]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  Sheng Chen,et al.  A clustering technique for digital communications channel equalization using radial basis function networks , 1993, IEEE Trans. Neural Networks.

[68]  Tianwen Zhang,et al.  A Full-Level Context Squeeze-and-Excitation ROI Extractor for SAR Ship Instance Segmentation , 2022, IEEE Geoscience and Remote Sensing Letters.

[69]  Guoqing Zhou,et al.  Study on Pixel Entanglement Theory for Imagery Classification , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[70]  Ruiqi Zhao,et al.  Isolated Ni atoms induced edge stabilities and equilibrium shapes of CVD-prepared hexagonal boron nitride on Ni(111) surface , 2022, New Journal of Chemistry.

[71]  Xiaoling Zhang,et al.  A Lightweight Adaptive RoI Extraction Network for Precise Aerial Image Instance Segmentation , 2021, IEEE Transactions on Instrumentation and Measurement.

[72]  Boli Xiong,et al.  An Anchor-Free Detection Method for Ship Targets in High-Resolution SAR Images , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[73]  Xiaoling Zhang,et al.  Injection of Traditional Hand-Crafted Features into Modern CNN-Based Models for SAR Ship Classification: What, Why, Where, and How , 2021, Remote. Sens..

[74]  Chunlei Huo,et al.  Multitask Learning for Ship Detection From Synthetic Aperture Radar Images , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[75]  Chuan He,et al.  MSARN: A Deep Neural Network Based on an Adaptive Recalibration Mechanism for Multiscale and Arbitrary-Oriented SAR Ship Detection , 2019, IEEE Access.

[76]  Christopher K. I. Williams,et al.  International Journal of Computer Vision manuscript No. (will be inserted by the editor) The PASCAL Visual Object Classes (VOC) Challenge , 2022 .