Subtask Attention Based Object Detection in Remote Sensing Images

Object detection in remote sensing images (RSIs) is one of the basic tasks in the field of remote sensing image automatic interpretation. In recent years, the deep object detection frameworks of natural scene images (NSIs) have been introduced into object detection on RSIs, and the detection performance has improved significantly because of the powerful feature representation. However, there are still many challenges concerning the particularities of remote sensing objects. One of the main challenges is the missed detection of small objects which have less than five percent of the pixels of the big objects. Generally, the existing algorithms choose to deal with this problem by multi-scale feature fusion based on a feature pyramid. However, the benefits of this strategy are limited, considering that the location of small objects in the feature map will disappear when the detection task is processed at the end of the network. In this study, we propose a subtask attention network (StAN), which handles the detection task directly on the shallow layer of the network. First, StAN contains one shared feature branch and two subtask attention branches of a semantic auxiliary subtask and a detection subtask based on the multi-task attention network (MTAN). Second, the detection branch uses only low-level features considering small objects. Third, the attention map guidance mechanism is put forward to optimize the network for keeping the identification ability. Fourth, the multi-dimensional sampling module (MdS), global multi-view channel weights (GMulW) and target-guided pixel attention (TPA) are designed for further improvement of the detection accuracy in complex scenes. The experimental results on the NWPU VHR-10 dataset and DOTA dataset demonstrated that the proposed algorithm achieved the SOTA performance, and the missed detection of small objects decreased. On the other hand, ablation experiments also proved the effects of MdS, GMulW and TPA.

[1]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[2]  Zhao Lin,et al.  Contextual Region-Based Convolutional Neural Network with Multilayer Fusion for SAR Ship Detection , 2017, Remote. Sens..

[3]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[4]  Xin Xu,et al.  Deformable ConvNet with Aspect Ratio Constrained NMS for Object Detection in Remote Sensing Imagery , 2017, Remote. Sens..

[5]  Junwei Han,et al.  Object detection in remote sensing imagery using a discriminatively trained mixture model , 2013 .

[6]  Lei Liu,et al.  Learning a Rotation Invariant Detector with Rotatable Bounding Box , 2017, ArXiv.

[7]  Gui-Song Xia,et al.  Accurate Annotation of Remote Sensing Images via Active Spectral Clustering with Little Expert Knowledge , 2015, Remote. Sens..

[8]  Xu Liu,et al.  Deep Adaptive Proposal Network for Object Detection in Optical Remote Sensing Images , 2018, ArXiv.

[9]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[10]  Gui-Song Xia,et al.  Learning RoI Transformer for Detecting Oriented Objects in Aerial Images , 2018, ArXiv.

[11]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[12]  Serkan Ozturk,et al.  A subclass supported convolutional neural network for object detection and localization in remote-sensing images , 2019, International Journal of Remote Sensing.

[13]  Xin Huang,et al.  Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[14]  Bo Li,et al.  Ship Detection in High-Resolution Optical Imagery Based on Anomaly Detector and Local Shape Feature , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[16]  Liangpei Zhang,et al.  An Efficient and Robust Integrated Geospatial Object Detection Framework for High Spatial Resolution Remote Sensing Imagery , 2017, Remote. Sens..

[17]  Gellért Máttyus,et al.  Fast Multiclass Vehicle Detection on Aerial Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[18]  Jun Zhang,et al.  Geospatial Object Detection in Remote Sensing Imagery Based on Multiscale Single-Shot Detector with Activated Semantics , 2018, Remote. Sens..

[19]  Yansheng Li,et al.  Accurate cloud detection in high-resolution remote sensing imagery by weakly supervised deep learning , 2020 .

[20]  Wenhui Diao,et al.  PCAN - Part-Based Context Attention Network for Thermal Power Plant Detection in Remote Sensing Imagery , 2021, Remote. Sens..

[21]  Lisha Cui,et al.  MDSSD: multi-scale deconvolutional single shot detector for small objects , 2018, Science China Information Sciences.

[22]  Yansheng Li,et al.  Image retrieval from remote sensing big data: A survey , 2021, Inf. Fusion.

[23]  Lin Lei,et al.  Vehicle Detection in Aerial Images Based on Region Convolutional Neural Networks and Hard Negative Example Mining , 2017, Sensors.

[24]  Adam Van Etten,et al.  You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery , 2018, ArXiv.

[25]  Shunping Xiao,et al.  Deformable Faster R-CNN with Aggregating Multi-Layer Features for Partially Occluded Object Detection in Optical Remote Sensing Images , 2018, Remote. Sens..

[26]  Junwei Han,et al.  Automatic landslide detection from remote-sensing imagery using a scene classification method based on BoVW and pLSA , 2013 .

[27]  Lei Guo,et al.  A coarse-to-fine model for airport detection from remote sensing images using target-oriented visual saliency and CRF , 2015, Neurocomputing.

[28]  Deren Li,et al.  Object Classification of Aerial Images With Bag-of-Visual Words , 2010, IEEE Geoscience and Remote Sensing Letters.

[29]  Yongjun Zhang,et al.  Robust infrared small target detection using local steering kernel reconstruction , 2018, Pattern Recognit..

[30]  Yansheng Li,et al.  Learning deep semantic segmentation network under multiple weakly-supervised constraints for cross-domain remote sensing image semantic segmentation , 2021 .

[31]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[32]  Liang Chen,et al.  Ship Detection for Optical Remote Sensing Images Based on Visual Attention Enhanced Network , 2019, Sensors.

[33]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.