Towards Automated Ship Detection and Category Recognition from High-Resolution Aerial Images

Ship category classification in high-resolution aerial images has attracted great interest in applications such as maritime security, naval construction, and port management. However, the applications of previous methods were mainly limited by the following issues: (i) The existing ship category classification methods were mainly to classify on accurately-cropped image patches. This is unsatisfactory for the results of the existing methods in practical applications, because the location of the ship in the patch obtained by the object detection varies greatly. (ii) The factors such as target scale variations and class imbalance have a great influence on the performance of ship category classification. Aiming at the issues above, we propose a novel ship detection and category classification framework. The category classification is based on accurate location. The detection network can generate more precise rotated bounding boxes in large-scale aerial images by introducing a novel Sequence Local Context (SLC) module. Besides, three different ship category classification networks are proposed to eliminate the effect of scale variations, and the Spatial Transform Crop (STC) operation is used to get aligned image patches. Whatever the problem of insufficient samples or class imbalance have, the Proposals Simulation Generator (PSG) is considered to handle this properly. Most remarkably, the state-of-the-art performance of our framework is demonstrated by experiments based on the 19-class ship dataset HRSC2016 and our multiclass warship dataset.

[1]  David A. Plaisted,et al.  A Heuristic Triangulation Algorithm , 1987, J. Algorithms.

[2]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Yiping Yang,et al.  Ship Rotated Bounding Box Space for Ship Extraction From High-Resolution Optical Satellite Images With Complex Backgrounds , 2016, IEEE Geoscience and Remote Sensing Letters.

[4]  Wenxian Yu,et al.  Toward Arbitrary-Oriented Ship Detection With Rotated Region Proposal and Discrimination Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[5]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[6]  Shuicheng Yan,et al.  A survey on deep learning-based fine-grained object classification and semantic segmentation , 2017, International Journal of Automation and Computing.

[7]  Xiao Xiang Zhu,et al.  HSF-Net: Multiscale Deep Feature Embedding for Ship Detection in Optical Remote Sensing Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Giancarlo Rufino,et al.  Wake Component Detection in X-Band SAR Images for Ship Heading and Velocity Estimation , 2016, Remote. Sens..

[9]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Zhenwei Shi,et al.  Ship Detection in Spaceborne Optical Image With SVD Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Filippo Biondi,et al.  A Polarimetric Extension of Low-Rank Plus Sparse Decomposition and Radon Transform for Ship Wake Detection in Synthetic Aperture Radar Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[12]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Xue Gao,et al.  An Improved Algorithm for Ship Target Detection in SAR Images Based on Faster R-CNN , 2018, 2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP).

[14]  Pasquale Iervolino,et al.  A new GLRT-based ship detection technique in SAR images , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[15]  Hong Zhang,et al.  Merchant Vessel Classification Based on Scattering Component Analysis for COSMO-SkyMed SAR Images , 2013, IEEE Geoscience and Remote Sensing Letters.

[16]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[17]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[18]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Junjie Wu,et al.  An Optimal 2-D Spectrum Matching Method for SAR Ground Moving Target Imaging , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Quentin Oliveau,et al.  Learning Attribute Representations for Remote Sensing Ship Category Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[21]  Jun Wu,et al.  A novel ship classification approach for high resolution SAR images based on the BDA-KELM classification model , 2017 .

[22]  Menglong Yan,et al.  Ship Instance Segmentation from Remote Sensing Images Using Sequence Local Context Module , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[23]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Xiangyang Xue,et al.  Arbitrary-Oriented Scene Text Detection via Rotation Proposals , 2017, IEEE Transactions on Multimedia.

[25]  Marco Grasso,et al.  Performance Analysis of Ship Wake Detection on Sentinel-1 SAR Images , 2017, Remote. Sens..

[26]  Yiping Yang,et al.  Rotated region based CNN for ship detection , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[27]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[28]  Filippo Biondi,et al.  Low-Rank Plus Sparse Decomposition and Localized Radon Transform for Ship-Wake Detection in Synthetic Aperture Radar Images , 2018, IEEE Geoscience and Remote Sensing Letters.

[29]  Domenico Velotto,et al.  Ship Classification in TerraSAR-X Images With Convolutional Neural Networks , 2018, IEEE Journal of Oceanic Engineering.

[30]  Ke Li,et al.  Rotation-Insensitive and Context-Augmented Object Detection in Remote Sensing Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Junghoon Seo,et al.  RBox-CNN: rotated bounding box based CNN for ship detection in remote sensing image , 2018, SIGSPATIAL/GIS.

[33]  Pasquale Iervolino,et al.  A Novel Ship Detector Based on the Generalized-Likelihood Ratio Test for SAR Imagery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[34]  Huanxin Zou,et al.  Ship Classification in TerraSAR-X Images With Feature Space Based Sparse Representation , 2013, IEEE Geoscience and Remote Sensing Letters.

[35]  Wei Liu,et al.  Ship Classification and Detection Based on CNN Using GF-3 SAR Images , 2018, Remote. Sens..

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Carmine Clemente,et al.  A Multifamily GLRT for Oil Spill Detection , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[39]  Yiping Yang,et al.  A High Resolution Optical Satellite Image Dataset for Ship Recognition and Some New Baselines , 2017, ICPRAM.

[40]  Filippo Biondi.  COSMO-SkyMed Staring Spotlight SAR Data for Micro-Motion and Inclination Angle Estimation of Ships by Pixel Tracking and Convex Optimization , 2019, Remote. Sens..

[41]  Guoquan Huang,et al.  Ship detection based on squeeze excitation skip-connection path networks for optical remote sensing images , 2019, Neurocomputing.

[42]  Yi Yang,et al.  Adversarial Complementary Learning for Weakly Supervised Object Localization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Zhiguo Jiang,et al.  Ship detection in optical remote sensing images based on deep convolutional neural networks , 2017 .

[45]  Yang Li,et al.  A Ship Rotation Detection Model in Remote Sensing Images Based on Feature Fusion Pyramid Network and Deep Reinforcement Learning , 2018, Remote. Sens..

[46]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Bin Deng,et al.  Generalised likelihood ratio test detector for micro motion targets in synthetic aperture radar raw signals , 2011 .

[48]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Biao Li,et al.  Multiscale Rotated Bounding Box-Based Deep Learning Method for Detecting Ship Targets in Remote Sensing Images , 2018, Sensors.

[50]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[51]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[52]  Gui-Song Xia,et al.  Rotation-Sensitive Regression for Oriented Scene Text Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[53]  Filippo Biondi.  Low rank plus sparse decomposition of synthetic aperture radar data for maritime surveillance , 2016, 2016 4th International Workshop on Compressed Sensing Theory and its Applications to Radar, Sonar and Remote Sensing (CoSeRa).

[54]  Michael S. Bernstein,et al.  Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations , 2016, International Journal of Computer Vision.