Dense Attention Fluid Network for Salient Object Detection in Optical Remote Sensing Images

Despite the remarkable advances in visual saliency analysis for natural scene images (NSIs), salient object detection (SOD) for optical remote sensing images (RSIs) still remains an open and challenging problem. In this paper, we propose an end-to-end Dense Attention Fluid Network (DAFNet) for SOD in optical RSIs. A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships, and is further embedded in a Dense Attention Fluid (DAF) structure that enables shallow attention cues flow into deep layers to guide the generation of high-level feature attention maps. Specifically, the GCA module is composed of two key components, where the global feature aggregation module achieves mutual reinforcement of salient feature embeddings from any two spatial locations, and the cascaded pyramid attention module tackles the scale variation issue by building up a cascaded pyramid framework to progressively refine the attention map in a coarse-to-fine manner. In addition, we construct a new and challenging optical RSI dataset for SOD that contains 2,000 images with pixel-wise saliency annotations, which is currently the largest publicly available benchmark. Extensive experiments demonstrate that our proposed DAFNet significantly outperforms the existing state-of-the-art SOD competitors. https://github.com/rmcong/DAFNet_TIP20

[1]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Xuelong Li,et al.  A Review of Co-Saliency Detection Algorithms , 2018 .

[3]  Qingming Huang,et al.  Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model , 2020, IEEE Transactions on Cybernetics.

[4]  Runmin Cong,et al.  RGB-D Salient Object Detection with Cross-Modality Modulation and Selection , 2020, ECCV.

[5]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[6]  Zhiguo Jiang,et al.  Unsupervised Saliency Model with Color Markov Chain for Oil Tank Detection , 2019, Remote. Sens..

[7]  David Dagan Feng,et al.  Robust saliency detection via regularized random walks ranking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jue Zhang,et al.  A New Saliency-Driven Fusion Method Based on Complex Wavelet Transform for Remote Sensing Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[9]  Fang Xu,et al.  Ship Detection in Optical Remote Sensing Images Based on Saliency and a Rotation-Invariant Descriptor , 2018, Remote. Sens..

[10]  Zhiguo Jiang,et al.  Sparsity-guided saliency detection for remote sensing images , 2015 .

[11]  Qingming Huang,et al.  Video Saliency Detection via Sparsity-Based Reconstruction and Propagation , 2019, IEEE Transactions on Image Processing.

[12]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chao Gao,et al.  BASNet: Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Qingming Huang,et al.  An Iterative Co-Saliency Framework for RGBD Images , 2017, IEEE Transactions on Cybernetics.

[16]  Yue Liu,et al.  Airport Extraction via Complementary Saliency Analysis and Saliency-Oriented Active Contour Model , 2018, IEEE Geoscience and Remote Sensing Letters.

[17]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[18]  Rynson W. H. Lau,et al.  Inferring Attention Shift Ranks of Objects for Image Saliency , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Yu-Wing Tai,et al.  Salient Region Detection via High-Dimensional Color Transform and Local Spatial Support , 2014, IEEE Transactions on Image Processing.

[20]  Hui Xu,et al.  Co-Saliency Detection via Hierarchical Consistency Measure , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[21]  Q. M. Jonathan Wu,et al.  Salient object detection via multi-scale attention CNN , 2018, Neurocomputing.

[22]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[23]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Sam Kwong,et al.  Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Qingming Huang,et al.  HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images , 2018, IEEE Transactions on Multimedia.

[27]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[28]  Chi-Wing Fu,et al.  Recurrently Aggregating Deep Features for Salient Object Detection , 2018, AAAI.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[31]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Qingming Huang,et al.  Global Context-Aware Progressive Aggregation Network for Salient Object Detection , 2020, AAAI.

[33]  Qingming Huang,et al.  Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion , 2016, IEEE Signal Processing Letters.

[34]  Xiaopeng Zhang,et al.  Building Extraction from Remotely Sensed Images by Integrating Saliency Cue , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Qingming Huang,et al.  Label Decoupling Framework for Salient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Bin Du,et al.  Region-of-Interest Detection via Superpixel-to-Pixel Saliency Analysis for Remote Sensing Image , 2016, IEEE Geoscience and Remote Sensing Letters.

[38]  Junhui Hou,et al.  CoADNet: Collaborative Aggregation-and-Distribution Networks for Co-Salient Object Detection , 2020, NeurIPS.

[39]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[41]  Jianmin Jiang,et al.  A Simple Pooling-Based Design for Real-Time Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Kenli Li,et al.  Hyperspectral Anomaly Detection With Attribute and Edge-Preserving Filters , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Qingming Huang,et al.  Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation , 2017, IEEE Transactions on Image Processing.

[44]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Yuanyuan Liu,et al.  Deep Salient Feature Based Anti-Noise Transfer Network for Scene Classification of Remote Sensing Imagery , 2018, Remote. Sens..

[46]  Qingming Huang,et al.  ASIF-Net: Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection , 2020, IEEE Transactions on Cybernetics.

[47]  Ali Borji,et al.  Salient object detection: A survey , 2014, Computational Visual Media.

[48]  Kai Zhao,et al.  Res2Net: A New Multi-Scale Backbone Architecture , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Yanan Liu,et al.  Saliency detection based on self-adaptive multiple feature fusion for remote sensing images , 2019, International Journal of Remote Sensing.

[50]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Ming-Ming Cheng,et al.  EGNet: Edge Guidance Network for Salient Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[52]  Weidong Cai,et al.  Reversion Correction and Regularized Random Walk Ranking for Saliency Detection , 2018, IEEE Transactions on Image Processing.

[53]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[54]  Gang Wang,et al.  Progressive Attention Guided Recurrent Network for Salient Object Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[55]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2016, IEEE transactions on pattern analysis and machine intelligence.

[56]  Dewen Hu,et al.  Salient Region Detection Using Diffusion Process on a Two-Layer Sparse Graph , 2017, IEEE Transactions on Image Processing.

[57]  Trung-Nghia Le,et al.  Video Salient Object Detection Using Spatiotemporal Deep Features , 2017, IEEE Transactions on Image Processing.

[58]  Xinwei Zheng,et al.  Efficient Saliency-Based Object Detection in Remote Sensing Images Using Deep Belief Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[59]  Jianhuang Lai,et al.  Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Steven C. H. Hoi,et al.  Salient Object Detection With Pyramid Attention and Salient Edges , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Guoqiang Han,et al.  R³Net: Recurrent Residual Refinement Network for Saliency Detection , 2018, IJCAI.

[62]  Hamidreza Sadreazami,et al.  Graph-Based Salient Object Detection using Background and Foreground Connectivity Cues , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[63]  Chunjie Zhang,et al.  A Parallel Down-Up Fusion Network for Salient Object Detection in Optical Remote Sensing Images , 2020, Neurocomputing.