ℱ3-Net: Feature Fusion and Filtration Network for Object Detection in Optical Remote Sensing Images

Object detection in remote sensing (RS) images is a challenging task due to the difficulties of small size, varied appearance, and complex background. Although a lot of methods have been developed to address this problem, many of them cannot fully exploit multilevel context information or handle cluttered background in RS images either. To this end, in this paper, we propose a feature fusion and filtration network (F3-Net) to improve object detection in RS images, which has higher capacity of combining the context information at multiple scales while suppressing the interference from the background. Specifically, F3-Net leverages a feature adaptation block with a residual structure to adjust the backbone network in an end-to-end manner, better considering the characteristics of RS images. Afterward, the network learns the context information of the object at multiple scales by hierarchically fusing the feature maps from different layers. In order to suppress the interference from cluttered background, the fused feature is then projected into a low-dimensional subspace by an additional feature filtration module. As a result, more relevant and accurate context information is extracted for further detection. Extensive experiments on DOTA, NWPU VHR-10, and UCAS AOD datasets demonstrate that the proposed detector achieves very promising detection performance.

[1]  Jun Zhou,et al.  Multiscale Visual Attention Networks for Object Detection in VHR Remote Sensing Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[2]  Zhihai Xu,et al.  $\mathcal{R}^2$ -CNN: Fast Tiny Object Detection in Large-Scale Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Peng Sun,et al.  Adaptive Saliency Biased Loss for Object Detection in Aerial Images , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Guang Shi,et al.  Bayesian Transfer Learning for Object Detection in Optical Remote Sensing Images , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Dazhuan Xu,et al.  Sig-NMS-Based Faster R-CNN Combining Transfer Learning for Small Target Detection in VHR Optical Remote Sensing Imagery , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Jue Wang,et al.  Detection of Multiclass Objects in Optical Remote Sensing Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[7]  Ke Li,et al.  Rotation-Insensitive and Context-Augmented Object Detection in Remote Sensing Images , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Ye Zhang,et al.  A light and faster regional convolutional neural network for object detection in optical remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[9]  Xiangyang Xue,et al.  Arbitrary-Oriented Scene Text Detection via Rotation Proposals , 2017, IEEE Transactions on Multimedia.

[10]  Dong Xu,et al.  Learning Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection , 2019, IEEE Transactions on Image Processing.

[11]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[12]  Shijian Lu,et al.  CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Xin Xu,et al.  Deformable ConvNet with Aspect Ratio Constrained NMS for Object Detection in Remote Sensing Imagery , 2017, Remote. Sens..

[14]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Kun Fu,et al.  FMSSD: Feature-Merged Single-Shot Detection for Multiscale Objects in Large-Scale Remote Sensing Imagery , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Junwei Han,et al.  Multi-class geospatial object detection and geographic image classification based on collection of part detectors , 2014 .

[17]  Jun Du,et al.  Adaptive Period Embedding for Representing Oriented Objects in Aerial Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Haigang Sui,et al.  Context-Aware Convolutional Neural Network for Object Detection in VHR Remote Sensing Imagery , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Jie Yang,et al.  Superpixel Segmentation of Polarimetric Synthetic Aperture Radar (SAR) Images Based on Generalized Mean Shift , 2018, Remote. Sens..

[20]  Ugur Halici,et al.  Texture-Based Airport Runway Detection , 2013, IEEE Geoscience and Remote Sensing Letters.

[21]  Bin Song,et al.  A Rotational Libra R-CNN Method for Ship Detection , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Yanli Wang,et al.  Object Detection in High Resolution Remote Sensing Imagery Based on Convolutional Neural Networks With Suitable Object Scale Features , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Alan M. Braga,et al.  A Median Regularized Level Set for Hierarchical Segmentation of SAR Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[24]  Yue Zhang,et al.  Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images , 2020 .

[25]  Nikos Koutsias,et al.  SVM-Based Fuzzy Decision Trees for Classification of High Spatial Resolution Remote Sensing Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Shunping Xiao,et al.  Deformable Faster R-CNN with Aggregating Multi-Layer Features for Partially Occluded Object Detection in Optical Remote Sensing Images , 2018, Remote. Sens..

[27]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[28]  Junchi Yan,et al.  R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object , 2019, AAAI.

[29]  Zhiqiang He,et al.  Cascaded Detection Framework Based on a Novel Backbone Network and Feature Fusion , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Jie Chen,et al.  Multi-Scale Spatial and Channel-wise Attention for Improving Object Detection in Remote Sensing Imagery , 2020, IEEE Geoscience and Remote Sensing Letters.

[31]  Xiaoqiang Lu,et al.  Gated and Axis-Concentrated Localization Network for Remote Sensing Object Detection , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Marcin Ciecholewski,et al.  River channel segmentation in polarimetric SAR images: Watershed transform combined with average contrast maximisation , 2017, Expert Syst. Appl..

[33]  Xu Tang,et al.  Progressively Refined Face Detection Through Semantics-Enriched Representation Learning , 2020, IEEE Transactions on Information Forensics and Security.

[34]  Jian Yang,et al.  Level Set Segmentation Algorithm for High-Resolution Polarimetric SAR Images Based on a Heterogeneous Clutter Model , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.