Learning Point-Guided Localization for Detection in Remote Sensing Images

Object detection in remote sensing images is challenging due to the dense distribution and arbitrary angle of the objects. It is a consensus that the oriented bounding box (OBB) is more suitable to represent the aerial objects. However, there are some extreme cases in regression-based OBB detection that make the regression target discontinuous, resulting in the poor performance. In this article, an analysis of the formats of OBB and the problems in its regression is presented, following with an exploration of transform localization from regression to keypoint estimation, which could be applied to avoid the problem of discontinuous regression target. Our novel method is called Object-wise Point-guided Localization Detector (OPLD). Continuously, a new prediction of center-point is introduced to refine the results, as the truncation problem caused by the cut graph. Lastly, in order to figure the problem of inconsistency between the localization quality and the classification score, both the endpoint scores and the classification score are adopted weighting as a result score. Experimental results are based on two widely used datasets, i.e., DOTA and HRSC2016. OPLD achieve 76.43% mAP and 78.35% mAP in OBB and horizontal bounding boxes tasks of DOTA-v1.0, which achieves state-of-the-art performance, respectively. Project page at https://github.com/yf19970118/OPLD-Pytorch.

[1]  Tong Zhang,et al.  Feature-Attentioned Object Detection in Remote Sensing Imagery , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[2]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Yue Zhang,et al.  SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Kun Fu,et al.  FMSSD: Feature-Merged Single-Shot Detection for Multiscale Objects in Large-Scale Remote Sensing Imagery , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Kai Chen,et al.  Gliding vertex on the horizontal bounding box for multi-oriented object detection , 2020, IEEE transactions on pattern analysis and machine intelligence.

[7]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Shijian Lu,et al.  CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Hao Chen,et al.  FCOS: Fully Convolutional One-Stage Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Wen Yang,et al.  Mask OBB: A Semantic Attention-Based Mask Oriented Bounding Box Representation for Multi-Category Object Detection in Aerial Images , 2019, Remote. Sens..

[13]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[15]  Shuchang Zhou,et al.  EAST: An Efficient and Accurate Scene Text Detector , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Kai Chen,et al.  Region Proposal by Guided Anchoring , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Yiping Yang,et al.  A High Resolution Optical Satellite Image Dataset for Ship Recognition and Some New Baselines , 2017, ICPRAM.

[19]  Ming Jiang,et al.  Parsing R-CNN for Instance-Level Human Analysis , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Wenxian Yu,et al.  Toward Arbitrary-Oriented Ship Detection With Rotated Region Proposal and Discrimination Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[22]  Jun Du,et al.  Adaptive Period Embedding for Representing Oriented Objects in Aerial Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Shifeng Zhang,et al.  Single-Shot Refinement Neural Network for Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Qi Tian,et al.  CenterNet: Keypoint Triplets for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[25]  Zhiqiang He,et al.  Cascaded Detection Framework Based on a Novel Backbone Network and Feature Fusion , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Zhaoxiang Zhang,et al.  Scale-Aware Trident Networks for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Lu Yang,et al.  Attention Inspiring Receptive-Fields Network for Learning Invariant Representations , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Yiping Yang,et al.  Rotated region based CNN for ship detection , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[29]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[30]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[32]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[34]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Menglong Yan,et al.  IoU-Adaptive Deformable R-CNN: Make Full Use of IoU for Multi-Class Object Detection in Remote Sensing Imagery , 2019, Remote. Sens..

[36]  Yue Zhang,et al.  SARD: Towards Scale-Aware Rotated Object Detection in Aerial Imagery , 2019, IEEE Access.

[37]  Nuno Vasconcelos,et al.  Cascade R-CNN: Delving Into High Quality Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Junjie Yan,et al.  Grid R-CNN , 2018, 1811.12030.

[39]  Ke Li,et al.  Rotation-Insensitive and Context-Augmented Object Detection in Remote Sensing Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Lu Yang,et al.  Renovating Parsing R-CNN for Accurate Multiple Human Parsing , 2020, ECCV.

[41]  Lin Lei,et al.  Multi-scale object detection in remote sensing imagery with convolutional neural networks , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[42]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, ECCV.

[43]  Lars Petersson,et al.  Improving Object Localization with Fitness NMS and Bounded IoU Loss , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Peter Reinartz,et al.  Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery , 2018, ACCV.

[45]  Lu Yang,et al.  Semantic Segmentation for High Spatial Resolution Remote Sensing Images Based on Convolution Neural Network and Pyramid Pooling Module , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[46]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[47]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[48]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[50]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[52]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Zhihui Wang,et al.  CPM R-CNN: Calibrating Point-guided Misalignment in Object Detection , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[54]  Xingyi Zhou,et al.  Bottom-Up Object Detection by Grouping Extreme and Center Points , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Xiangyang Xue,et al.  Arbitrary-Oriented Scene Text Detection via Rotation Proposals , 2017, IEEE Transactions on Multimedia.

[56]  Yuning Jiang,et al.  Acquisition of Localization Confidence for Accurate Object Detection , 2018, ECCV.

[57]  Gui-Song Xia,et al.  Learning RoI Transformer for Detecting Oriented Objects in Aerial Images , 2018, ArXiv.

[58]  Fuchun Sun,et al.  HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Stephen Lin,et al.  RepPoints: Point Set Representation for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[60]  Xiaohua Tong,et al.  An Optimized Deep Neural Network Detecting Small and Narrow Rectangular Objects in Google Earth Images , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.