Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images

Despite the remarkable advances in visual saliency analysis for natural scene images (NSIs), salient object detection (SOD) for optical remote sensing images (RSIs) still remains an open and challenging problem. In this paper, we propose an end-to-end Dense Attention Fluid Network (DAFNet) for SOD in optical RSIs. A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships, and is further embedded in a Dense Attention Fluid (DAF) structure that enables shallow attention cues flow into deep layers to guide the generation of high-level feature attention maps. Specifically, the GCA module is composed of two key components, where the global feature aggregation module achieves mutual reinforcement of salient feature embeddings from any two spatial locations, and the cascaded pyramid attention module tackles the scale variation issue by building up a cascaded pyramid framework to progressively refine the attention map in a coarse-to-fine manner. In addition, we construct a new and challenging optical RSI dataset for SOD that contains 2,000 images with pixel-wise saliency annotations, which is currently the largest publicly available benchmark. Extensive experiments demonstrate that our proposed DAFNet significantly outperforms the existing state-of-the-art SOD competitors. https://github.com/rmcong/DAFNet_TIP20

[1]  Qingming Huang,et al.  HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images , 2018, IEEE Transactions on Multimedia.

[2]  Xuelong Li,et al.  A Review of Co-Saliency Detection Algorithms , 2018 .

[3]  Yuanyuan Liu,et al.  Deep Salient Feature Based Anti-Noise Transfer Network for Scene Classification of Remote Sensing Imagery , 2018, Remote. Sens..

[4]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[6]  Kai Zhao,et al.  Res2Net: A New Multi-Scale Backbone Architecture , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Trung-Nghia Le,et al.  Video Salient Object Detection Using Spatiotemporal Deep Features , 2017, IEEE Transactions on Image Processing.

[8]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[9]  Qingming Huang,et al.  Global Context-Aware Progressive Aggregation Network for Salient Object Detection , 2020, AAAI.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Qingming Huang,et al.  Label Decoupling Framework for Salient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Weidong Cai,et al.  Reversion Correction and Regularized Random Walk Ranking for Saliency Detection , 2018, IEEE Transactions on Image Processing.

[13]  Yu-Wing Tai,et al.  Salient Region Detection via High-Dimensional Color Transform and Local Spatial Support , 2014, IEEE Transactions on Image Processing.

[14]  Qingming Huang,et al.  Video Saliency Detection via Sparsity-Based Reconstruction and Propagation , 2019, IEEE Transactions on Image Processing.

[15]  Hui Xu,et al.  Co-Saliency Detection via Hierarchical Consistency Measure , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[16]  David Dagan Feng,et al.  Robust saliency detection via regularized random walks ranking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ming-Ming Cheng,et al.  EGNet: Edge Guidance Network for Salient Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Zhiguo Jiang,et al.  Unsupervised Saliency Model with Color Markov Chain for Oil Tank Detection , 2019, Remote. Sens..

[20]  Xiaopeng Zhang,et al.  Building Extraction from Remotely Sensed Images by Integrating Saliency Cue , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[21]  Xinwei Zheng,et al.  Efficient Saliency-Based Object Detection in Remote Sensing Images Using Deep Belief Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[22]  Runmin Cong,et al.  RGB-D Salient Object Detection with Cross-Modality Modulation and Selection , 2020, ECCV.

[23]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Qingming Huang,et al.  ASIF-Net: Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection , 2020, IEEE Transactions on Cybernetics.

[25]  Rynson W. H. Lau,et al.  Inferring Attention Shift Ranks of Objects for Image Saliency , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yanan Liu,et al.  Saliency detection based on self-adaptive multiple feature fusion for remote sensing images , 2019, International Journal of Remote Sensing.

[28]  Q. M. Jonathan Wu,et al.  Salient object detection via multi-scale attention CNN , 2018, Neurocomputing.

[29]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[30]  Chao Gao,et al.  BASNet: Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Qingming Huang,et al.  Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation , 2017, IEEE Transactions on Image Processing.

[32]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[33]  Steven C. H. Hoi,et al.  Salient Object Detection With Pyramid Attention and Salient Edges , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Qingming Huang,et al.  Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion , 2016, IEEE Signal Processing Letters.

[35]  Qingming Huang,et al.  An Iterative Co-Saliency Framework for RGBD Images , 2017, IEEE Transactions on Cybernetics.

[36]  Dewen Hu,et al.  Salient Region Detection Using Diffusion Process on a Two-Layer Sparse Graph , 2017, IEEE Transactions on Image Processing.

[37]  Yue Liu,et al.  Airport Extraction via Complementary Saliency Analysis and Saliency-Oriented Active Contour Model , 2018, IEEE Geoscience and Remote Sensing Letters.

[38]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Bin Du,et al.  Region-of-Interest Detection via Superpixel-to-Pixel Saliency Analysis for Remote Sensing Image , 2016, IEEE Geoscience and Remote Sensing Letters.

[40]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2016, IEEE transactions on pattern analysis and machine intelligence.

[41]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Sam Kwong,et al.  Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Qingming Huang,et al.  Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model , 2020, IEEE Transactions on Cybernetics.

[45]  Junhui Hou,et al.  CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection , 2020, NeurIPS.

[46]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[47]  Jianmin Jiang,et al.  A Simple Pooling-Based Design for Real-Time Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Hamidreza Sadreazami,et al.  Graph-Based Salient Object Detection using Background and Foreground Connectivity Cues , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[49]  Chunjie Zhang,et al.  A Parallel Down-Up Fusion Network for Salient Object Detection in Optical Remote Sensing Images , 2020, Neurocomputing.

[50]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[51]  Fang Xu,et al.  Ship Detection in Optical Remote Sensing Images Based on Saliency and a Rotation-Invariant Descriptor , 2018, Remote. Sens..

[52]  Chi-Wing Fu,et al.  Recurrently Aggregating Deep Features for Salient Object Detection , 2018, AAAI.

[53]  Zhiguo Jiang,et al.  Sparsity-guided saliency detection for remote sensing images , 2015 .

[54]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[55]  Jianhuang Lai,et al.  Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[57]  Kenli Li,et al.  Hyperspectral Anomaly Detection With Attribute and Edge-Preserving Filters , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[58]  Guoqiang Han,et al.  R³Net: Recurrent Residual Refinement Network for Saliency Detection , 2018, IJCAI.

[59]  Jue Zhang,et al.  A New Saliency-Driven Fusion Method Based on Complex Wavelet Transform for Remote Sensing Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[60]  Gang Wang,et al.  Progressive Attention Guided Recurrent Network for Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[61]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[62]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).