RRNet: Relational Reasoning Network with Parallel Multi-scale Attention for Salient Object Detection in Optical Remote Sensing Images

Résumé—Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. Despite some saliency models were proposed to solve the intrinsic problem of optical RSIs (such as complex background and scale-variant objects), the accuracy and completeness are still unsatisfactory. To this end, we propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs in this paper. The relational reasoning module that integrates the spatial and the channel dimensions is designed to infer the semantic relationship by utilizing high-level encoder features, thereby promoting the generation of more complete detection results. The parallel multi-scale attention module is proposed to effectively restore the detail information and address the scale variation of salient objects by using the low-level features refined by multi-scale attention. Extensive experiments on two datasets demonstrate that our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively. https://rmcong.github.io/proj_RRNet.html.

[1]  Fang Xu,et al.  Ship Detection in Optical Remote Sensing Images Based on Saliency and a Rotation-Invariant Descriptor , 2018, Remote. Sens..

[2]  Xuelong Li,et al.  A Multiview-Based Parameter Free Framework for Group Detection , 2017, AAAI.

[3]  Qingming Huang,et al.  HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images , 2018, IEEE Transactions on Multimedia.

[4]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[5]  Xiaopeng Wei,et al.  Camouflaged Object Segmentation with Distraction Mining , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Qi Wang,et al.  Multitask Attention Network for Lane Detection and Fitting , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Qi Wang,et al.  VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection , 2019, IEEE Transactions on Image Processing.

[8]  Zhiguo Jiang,et al.  Unsupervised Saliency Model with Color Markov Chain for Oil Tank Detection , 2019, Remote. Sens..

[9]  Runmin Cong,et al.  RGB-D Salient Object Detection with Cross-Modality Modulation and Selection , 2020, ECCV.

[10]  Abhinav Gupta,et al.  Beyond Grids: Learning Graph Representations for Visual Recognition , 2018, NeurIPS.

[11]  Xiaochun Cao,et al.  Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images , 2020, IEEE Transactions on Image Processing.

[12]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[13]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Chunjie Zhang,et al.  A Parallel Down-Up Fusion Network for Salient Object Detection in Optical Remote Sensing Images , 2020, Neurocomputing.

[15]  Qingming Huang,et al.  Video Saliency Detection via Sparsity-Based Reconstruction and Propagation , 2019, IEEE Transactions on Image Processing.

[16]  Qingming Huang,et al.  ASIF-Net: Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection , 2020, IEEE Transactions on Cybernetics.

[17]  Qingming Huang,et al.  Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model , 2020, IEEE Transactions on Cybernetics.

[18]  Yanan Liu,et al.  Saliency detection based on self-adaptive multiple feature fusion for remote sensing images , 2019, International Journal of Remote Sensing.

[19]  Junhui Hou,et al.  CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection , 2020, NeurIPS.

[20]  Chi-Wing Fu,et al.  Recurrently Aggregating Deep Features for Salient Object Detection , 2018, AAAI.

[21]  Zhiguo Jiang,et al.  Sparsity-guided saliency detection for remote sensing images , 2015 .

[22]  Shuicheng Yan,et al.  Graph-Based Global Reasoning Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Qingming Huang,et al.  Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion , 2016, IEEE Signal Processing Letters.

[24]  Kai Zhao,et al.  Res2Net: A New Multi-Scale Backbone Architecture , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Bin Du,et al.  Region-of-Interest Detection via Superpixel-to-Pixel Saliency Analysis for Remote Sensing Image , 2016, IEEE Geoscience and Remote Sensing Letters.

[26]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[28]  Qingming Huang,et al.  An Iterative Co-Saliency Framework for RGBD Images , 2017, IEEE Transactions on Cybernetics.

[29]  Gang Wan,et al.  Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark , 2020, ISPRS Journal of Photogrammetry and Remote Sensing.

[30]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.

[31]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[32]  Zheng Lin,et al.  Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[34]  Kwan-Liu Ma,et al.  Stereoscopic Thumbnail Creation via Efficient Stereo Saliency Detection , 2017, IEEE Transactions on Visualization and Computer Graphics.

[35]  Huchuan Lu,et al.  Multi-Scale Interactive Network for Salient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Chenglizhao Chen,et al.  Mutual Graph Learning for Camouflaged Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Libao Zhang,et al.  Salient Object Detection Based on Progressively Supervised Learning for Remote Sensing Images , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Guoqiang Han,et al.  R³Net: Recurrent Residual Refinement Network for Saliency Detection , 2018, IJCAI.

[39]  Qingming Huang,et al.  Global Context-Aware Progressive Aggregation Network for Salient Object Detection , 2020, AAAI.

[40]  Jocelyn Chanussot,et al.  ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Ming-Ming Cheng,et al.  EGNet: Edge Guidance Network for Salient Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Libao Zhang,et al.  Region-of-Interest Extraction Based on Frequency Domain Analysis and Salient Region Detection for Remote Sensing Image , 2014, IEEE Geoscience and Remote Sensing Letters.

[43]  Fei Wang,et al.  PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Sam Kwong,et al.  Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Lei Zhang,et al.  Suppress and Balance: A Simple Gated Network for Salient Object Detection , 2020, ECCV.

[46]  Qingming Huang,et al.  F3Net: Fusion, Feedback and Focus for Salient Object Detection , 2019, AAAI.

[47]  Ye Zhang,et al.  Region search based on hybrid convolutional neural network in optical remote sensing images , 2019, Int. J. Distributed Sens. Networks.

[48]  Qingming Huang,et al.  Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation , 2017, IEEE Transactions on Image Processing.

[49]  Libao Zhang,et al.  Region of interest extraction based on multiscale visual saliency analysis for remote sensing images , 2015 .

[50]  Weisi Lin,et al.  Saliency-Guided Quality Assessment of Screen Content Images , 2016, IEEE Transactions on Multimedia.