A 2 RMNet : Adaptively Aspect Ratio Multi-Scale Network for Object Detection in Remote Sensing Images

Abstract: Object detection is a significant and challenging problem in the study area of remote sensing and image analysis. However, most existing methods are easy to miss or incorrectly locate objects due to the various sizes and aspect ratios of objects. In this paper, we propose a novel end-to-end Adaptively Aspect Ratio Multi-Scale Network (A2RMNet) to solve this problem. On the one hand, we design a multi-scale feature gate fusion network to adaptively integrate the multi-scale features of objects. This network is composed of gate fusion modules, refine blocks and region proposal networks. On the other hand, an aspect ratio attention network is leveraged to preserve the aspect ratios of objects, which alleviates the excessive shape distortions of objects caused by aspect ratio changes during training. Experiments show that the proposed A2RMNet significantly outperforms the previous state of the arts on the DOTA dataset, NWPU VHR-10 dataset, RSOD dataset and UCAS-AOD dataset by 5.73%, 7.06%, 3.27% and 2.24%, respectively.

[1]  Q. Mcnemar.  Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[2]  G. Foody.  Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy , 2004 .

[3]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Cem Ünsalan,et al.  Urban-Area and Building Detection Using SIFT Keypoints and Graph Theory , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[7]  Yu Li,et al.  Automatic Target Detection in High-Resolution Remote Sensing Images Using Spatial Sparse Coding Bag-of-Words Model , 2012, IEEE Geoscience and Remote Sensing Letters.

[8]  Bo Li,et al.  Ship Detection in High-Resolution Optical Imagery Based on Anomaly Detector and Local Shape Feature , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Shiming Xiang,et al.  Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks , 2014, IEEE Geoscience and Remote Sensing Letters.

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[12]  Lei Guo,et al.  Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[14]  Lei Guo,et al.  A coarse-to-fine model for airport detection from remote sensing images using target-oriented visual saliency and CRF , 2015, Neurocomputing.

[15]  Zhifeng Xiao,et al.  Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images , 2015 .

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  Qixiang Ye,et al.  Orientation robust object detection in aerial images using deep convolutional neural network , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[20]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[22]  Bin Zou,et al.  Scale- and Region-Dependence in Landscape-PM2.5 Correlation: Implications for Urban Planning , 2017, Remote. Sens..

[23]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Hannes Taubenböck,et al.  Unsupervised change detection in VHR remote sensing imagery - an object-based clustering approach in a dynamic urban environment , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[25]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Libao Zhang,et al.  Airport Detection and Aircraft Recognition Based on Two-Layer Saliency Model in High Spatial Resolution Remote-Sensing Images , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Liangpei Zhang,et al.  An Efficient and Robust Integrated Geospatial Object Detection Framework for High Spatial Resolution Remote Sensing Imagery , 2017, Remote. Sens..

[29]  Bahareh Kalantar,et al.  Multiple Moving Object Detection From UAV Videos Using Trajectories of Matched Regional Adjacency Graphs , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[31]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Yang Long,et al.  Airport Detection Based on a Multiscale Fusion Feature for Optical Remote Sensing Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[33]  Qing Liu,et al.  Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Jiao Jiao,et al.  A Densely Connected End-to-End Neural Network for Multiscale and Multiscene SAR Ship Detection , 2018, IEEE Access.

[36]  Yan Zhou,et al.  Fast Automatic Airport Detection in Remote Sensing Images Using Convolutional Neural Networks , 2018, Remote. Sens..

[37]  Ye Zhang,et al.  A light and faster regional convolutional neural network for object detection in optical remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[38]  Jiao Jiao,et al.  An End-to-End Neural Network for Road Extraction From Remote Sensing Imagery by Multiple Feature Pyramid Network , 2018, IEEE Access.

[39]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Xiao Xiang Zhu,et al.  Vehicle Instance Segmentation From Aerial Image and Video Using a Multitask Learning Residual Fully Convolutional Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Xiao Xiang Zhu,et al.  HSF-Net: Multiscale Deep Feature Embedding for Ship Detection in Optical Remote Sensing Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Peter Reinartz,et al.  Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery , 2018, ACCV.

[43]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[44]  Wei Guo,et al.  Geospatial Object Detection in High Resolution Satellite Images Based on Multi-Scale Convolutional Neural Network , 2018, Remote. Sens..

[45]  Menglong Yan,et al.  IoU-Adaptive Deformable R-CNN: Make Full Use of IoU for Multi-Class Object Detection in Remote Sensing Imagery , 2019, Remote. Sens..