Attention Mask R-CNN for Ship Detection and Segmentation From Remote Sensing Images

In recent years, ship detection in satellite remote sensing images has become an important research topic. Most existing methods detect ships by using a rectangular bounding box but do not perform segmentation down to the pixel level. This paper proposes a ship detection and segmentation method based on an improved Mask R-CNN model. Our proposed method can accurately detect and segment ships at the pixel level. By adding a bottom-up structure to the FPN structure of Mask R-CNN, the path between the lower layers and the topmost layer is shortened, allowing the lower layer features to be more effectively utilized at the top layer. In the bottom-up structure, we use channel-wise attention to assign weights in each channel and use the spatial attention mechanism to assign a corresponding weight at each pixel in the feature maps. This allows the feature maps to respond better to the target’s features. Using our method, the detection and segmentation mAPs increased from 70.6% and 62.0% to 76.1% and 65.8%, respectively.

[1]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Yongchao Gong,et al.  Mask Scoring R-CNN , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Huanxin Zou,et al.  A Bilateral CFAR Algorithm for Ship Detection in SAR Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[5]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[6]  Gui-Song Xia,et al.  Learning RoI Transformer for Detecting Oriented Objects in Aerial Images , 2018, ArXiv.

[7]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Zhao Lin,et al.  A modified faster R-CNN based on CFAR algorithm for SAR ship detection , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[9]  Zongjie Cao,et al.  SAR Target Recognition in Large Scene Images via Region-Based Convolutional Neural Networks , 2018, Remote. Sens..

[10]  Xiao Xiang Zhu,et al.  HSF-Net: Multiscale Deep Feature Embedding for Ship Detection in Optical Remote Sensing Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[11]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[12]  Liu Xin,et al.  On-board ship targets detection method based on multi-scale salience enhancement for remote sensing image , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[13]  Jie Huang,et al.  Region proposal for ship detection based on structured forests edge method , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[14]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[15]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Gao Xin,et al.  An Aircraft Detection Method Based on Convolutional Neural Networks in High-Resolution SAR Images , 2017 .

[17]  Lena Chang,et al.  Ship Detection Based on YOLOv2 for SAR Imagery , 2019, Remote. Sens..

[18]  Zhiguo Jiang,et al.  Inshore Ship Detection Based on Mask R-CNN , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[19]  Gaofeng Meng,et al.  FusionNet: Edge Aware Deep Convolutional Networks for Semantic Segmentation of Remote Sensing Harbor Images , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[22]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[23]  Shilin Zhou,et al.  Ship Detection Based on Complex Signal Kurtosis in Single-Channel SAR Imagery , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Tat-Seng Chua,et al.  SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Xiao Xiang Zhu,et al.  R3-Net: A Deep Network for Multioriented Vehicle Detection in Aerial Images and Videos , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Liang Chen,et al.  An Intensity-Space Domain CFAR Method for Ship Detection in HR SAR Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[27]  Zhao Lin,et al.  Contextual Region-Based Convolutional Neural Network with Multilayer Fusion for SAR Ship Detection , 2017, Remote. Sens..

[28]  Juho Kannala,et al.  Context Aware Query Image Representation for Particular Object Retrieval , 2017, SCIA.

[29]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[30]  Zhenwei Shi,et al.  Fully Convolutional Network With Task Partitioning for Inshore Ship Detection in Optical Remote Sensing Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[31]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Hu Lei,et al.  Fast ship detection from optical satellite images based on ship distribution probability analysis , 2016, 2016 4th International Workshop on Earth Observation and Remote Sensing Applications (EORSA).

[33]  Menglong Yan,et al.  R2CNN++: Multi-Dimensional Attention Based Rotation Invariant Detector with Robust Anchor Strategy , 2018, ArXiv.

[34]  Jiao Jiao,et al.  A Densely Connected End-to-End Neural Network for Multiscale and Multiscene SAR Ship Detection , 2018, IEEE Access.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Jianwei Li,et al.  Ship detection in SAR images based on an improved faster R-CNN , 2017, 2017 SAR in Big Data Era: Models, Methods and Applications (BIGSARDATA).

[37]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[38]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Xiaogang Wang,et al.  Multi-context Attention for Human Pose Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Wang Yan-hua,et al.  A real-time on-board ship targets detection method for optical remote sensing satellite , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[43]  Gangyao Kuang,et al.  Squeeze and Excitation Rank Faster R-CNN for Ship Detection in SAR Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[44]  Huanxin Zou,et al.  Area Ratio Invariant Feature Group for Ship Detection in SAR Imagery , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.