A Novel Effectively Optimized One-Stage Network for Object Detection in Remote Sensing Imagery

With great significance in military and civilian applications, the topic of detecting small and densely arranged objects in wide-scale remote sensing imagery is still challenging nowadays. To solve this problem, we propose a novel effectively optimized one-stage network (NEOON). As a fully convolutional network, NEOON consists of four parts: Feature extraction, feature fusion, feature enhancement, and multi-scale detection. To extract effective features, the first part has implemented bottom-up and top-down coherent processing by taking successive down-sampling and up-sampling operations in conjunction with residual modules. The second part consolidates high-level and low-level features by adopting concatenation operations with subsequent convolutional operations to explicitly yield strong feature representation and semantic information. The third part is implemented by constructing a receptive field enhancement (RFE) module and incorporating it into the fore part of the network where the information of small objects exists. The final part is achieved by four detectors with different sensitivities accessing the fused features, all four parallel, to enable the network to make full use of information of objects in different scales. Besides, the Focal Loss is set to enable the cross entropy for classification to solve the tough problem of class imbalance in one-stage methods. In addition, we introduce the Soft-NMS to preserve accurate bounding boxes in the post-processing stage especially for densely arranged objects. Note that the split and merge strategy and multi-scale training strategy are employed in training. Thorough experiments are performed on ACS datasets constructed by us and NWPU VHR-10 datasets to evaluate the performance of NEOON. Specifically, 4.77% and 5.50% improvements in mAP and recall, respectively, on the ACS dataset as compared to YOLOv3 powerfully prove that NEOON can effectually improve the detection accuracy of small objects in remote sensing imagery. In addition, extensive experiments and comprehensive evaluations on the NWPU VHR-10 dataset with 10 classes have illustrated the superiority of NEOON in the extraction of spatial information of high-resolution remote sensing images.

[1]  Xuelong Li,et al.  Locality Adaptive Discriminant Analysis for Spectral–Spatial Classification of Hyperspectral Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[2]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Liangpei Zhang,et al.  An Efficient and Robust Integrated Geospatial Object Detection Framework for High Spatial Resolution Remote Sensing Imagery , 2017, Remote. Sens..

[4]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[5]  Feng Yang,et al.  Ship Detection From Thermal Remote Sensing Imagery Through Region-Based Deep Forest , 2018, IEEE Geoscience and Remote Sensing Letters.

[6]  Wei Zhang,et al.  Deconv R-CNN for Small Object Detection on Remote Sensing Images , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[7]  He Chen,et al.  Arbitrary-Oriented Ship Detection Framework in Optical Remote-Sensing Images , 2018, IEEE Geoscience and Remote Sensing Letters.

[8]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jianxiong Xiao,et al.  R-CNN for Small Object Detection , 2016, ACCV.

[11]  Zhiguo Jiang,et al.  Online Exemplar-Based Fully Convolutional Network for Aircraft Detection in Remote Sensing Images , 2018, IEEE Geoscience and Remote Sensing Letters.

[12]  Suman Debnath,et al.  Numerical-Stiffness-Based Simulation of Mixed Transmission Systems , 2018, IEEE Transactions on Industrial Electronics.

[13]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[15]  Menglong Yan,et al.  Object Detection with Head Direction in Remote Sensing Images Based on Rotational Region CNN , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[16]  Gongjian Wen,et al.  Occluded Object Detection in High-Resolution Remote Sensing Images Using Partial Configuration Object Model , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[18]  Junyu Gao,et al.  Embedding structured contour and location prior in siamesed fully convolutional networks for road detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Rodrigo Capobianco Guido,et al.  Practical and Useful Tips on Discrete Wavelet Transforms [sp Tips & Tricks] , 2015, IEEE Signal Processing Magazine.

[20]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[22]  Ugur Halici,et al.  Deep convolutional neural networks for airport detection in remote sensing images , 2018, 2018 26th Signal Processing and Communications Applications Conference (SIU).

[23]  K.C. Hwang,et al.  A Modified Sierpinski Fractal Antenna for Multiband Application , 2007, IEEE Antennas and Wireless Propagation Letters.

[24]  Jun Zhang,et al.  Geospatial Object Detection in Remote Sensing Imagery Based on Multiscale Single-Shot Detector with Activated Semantics , 2018, Remote. Sens..

[25]  Xiao Xiang Zhu,et al.  Hierarchical Region Based Convolution Neural Network for Multiscale Object Detection in Remote Sensing Images , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[26]  Karanbir Singh Chahal,et al.  A Survey of Modern Object Detection Literature using Deep Learning , 2018, ArXiv.

[27]  Yunsong Li,et al.  High-quality spectral-spatial reconstruction using saliency detection and deep feature enhancement , 2019, Pattern Recognit..

[28]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[29]  Zhenyu Huang,et al.  Multiple Marginal Fisher Analysis , 2019, IEEE Transactions on Industrial Electronics.

[30]  Yanfei Zhong,et al.  Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery , 2018 .

[31]  Yunchao Wei,et al.  Perceptual Generative Adversarial Networks for Small Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Congcong Li,et al.  Visual Detail Augmented Mapping for Small Aerial Target Detection , 2019, Remote. Sens..

[33]  Adam Van Etten,et al.  You Only Look Twice: Rapid Multi-Scale Object Detection In Satellite Imagery , 2018, ArXiv.

[34]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, International Journal of Computer Vision.

[35]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[36]  Wei-Yun Yau,et al.  Structured AutoEncoders for Subspace Clustering , 2018, IEEE Transactions on Image Processing.

[37]  Yue Wu,et al.  A Novel Multi-Model Decision Fusion Network for Object Detection in Remote Sensing Images , 2019, Remote. Sens..

[38]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Emanuel Guariglia,et al.  Harmonic Sierpinski Gasket and Applications , 2018, Entropy.

[40]  Zhenwei Shi,et al.  Random Access Memories: A New Paradigm for Target Detection in High Resolution Aerial Remote Sensing Images , 2018, IEEE Transactions on Image Processing.

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Li Lei,et al.  Research on object detection technique in high resolution remote sensing images based on U-Net , 2018, 2018 Chinese Control And Decision Conference (CCDC).

[43]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Emanuel Guariglia,et al.  Entropy and Fractal Antennas , 2016, Entropy.

[45]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[46]  Kil To Chong,et al.  Object Detection in Very High-Resolution Aerial Images Using One-Stage Densely Connected Feature Pyramid Network , 2018, Sensors.

[47]  Qing Liu,et al.  Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[49]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Yunsong Li,et al.  Structure Tensor and Guided Filtering-Based Algorithm for Hyperspectral Anomaly Detection , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[52]  Yunhong Wang,et al.  Receptive Field Block Net for Accurate and Fast Object Detection , 2017, ECCV.

[53]  Ke Li,et al.  Rotation-Insensitive and Context-Augmented Object Detection in Remote Sensing Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Xiaohui Hu,et al.  Fast aircraft detection based on region locating network in large-scale remote sensing images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[55]  Wei Guo,et al.  Geospatial Object Detection in High Resolution Satellite Images Based on Multi-Scale Convolutional Neural Network , 2018, Remote. Sens..

[56]  Ye Zhang,et al.  A light and faster regional convolutional neural network for object detection in optical remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[57]  Zengzhou Hao,et al.  Edge-Guided Image Object Detection in Multiscale Segmentation for High-Resolution Remotely Sensed Imagery , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[58]  Lin Lei,et al.  An enhanced deep convolutional neural network for densely packed objects detection in remote sensing images , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[59]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[61]  Gang Wang,et al.  A Single Shot Framework with Multi-Scale Feature Fusion for Geospatial Object Detection , 2019, Remote. Sens..

[62]  Adrien Chan-Hon-Tong,et al.  Object Detection in Remote Sensing Images with Center Only , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[63]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Xiaodong Zhang,et al.  Geospatial Object Detection on High Resolution Remote Sensing Imagery Based on Double Multi-Scale Feature Pyramid Network , 2019, Remote. Sens..