A Parallel Down-Up Fusion Network for Salient Object Detection in Optical Remote Sensing Images

Abstract The diverse spatial resolutions, various object types, scales and orientations, and cluttered backgrounds in optical remote sensing images (RSIs) challenge the current salient object detection (SOD) approaches. It is commonly unsatisfactory to directly employ the SOD approaches designed for nature scene images (NSIs) to RSIs. In this paper, we propose a novel Parallel Down-up Fusion network (PDF-Net) for SOD in optical RSIs, which takes full advantage of the in-path low- and high-level features and cross-path multi-resolution features to distinguish diversely scaled salient objects and suppress the cluttered backgrounds. To be specific, keeping a key observation that the salient objects still are salient no matter the resolutions of images are in mind, the PDF-Net takes successive down-sampling to form five parallel paths and perceive scaled salient objects that are commonly existed in optical RSIs. Meanwhile, we adopt the dense connections to take advantage of both low- and high-level information in the same path and build up the relations of cross paths, which explicitly yield strong feature representations. At last, we fuse the multiple-resolution features in parallel paths to combine the benefits of the features with different resolutions, i.e., the high-resolution feature consisting of complete structure and clear details while the low-resolution features highlighting the scaled salient objects. Extensive experiments on the ORSSD dataset demonstrate that the proposed network is superior to the state-of-the-art approaches both qualitatively and quantitatively.

[1]  Haibin Ling,et al.  A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Fang Xu,et al.  Ship Detection from Optical Remote Sensing Images Using Multi-Scale Analysis and Fourier HOG Descriptor , 2019, Remote. Sens..

[3]  Dan Su,et al.  Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection , 2019, Pattern Recognit..

[4]  Adriano M. Pereira,et al.  A video summarization approach based on the emulation of bottom-up mechanisms of visual attention , 2017, Journal of Intelligent Information Systems.

[5]  Haibin Ling,et al.  Salient Object Detection in the Deep Learning Era: An In-Depth Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Qingming Huang,et al.  HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images , 2018, IEEE Transactions on Multimedia.

[7]  Qingming Huang,et al.  Video Saliency Detection via Sparsity-Based Reconstruction and Propagation , 2019, IEEE Transactions on Image Processing.

[8]  Xiaopeng Zhang,et al.  Building Extraction from Remotely Sensed Images by Integrating Saliency Cue , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[10]  Chi-Wing Fu,et al.  Recurrently Aggregating Deep Features for Salient Object Detection , 2018, AAAI.

[11]  Zhiguo Jiang,et al.  Sparsity-guided saliency detection for remote sensing images , 2015 .

[12]  Weisi Lin,et al.  Saliency-Guided Quality Assessment of Screen Content Images , 2016, IEEE Transactions on Multimedia.

[13]  Qi Tian,et al.  Saliency-Aware Nonparametric Foreground Annotation Based on Weakly Labeled Data , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Qingming Huang,et al.  Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation , 2017, IEEE Transactions on Image Processing.

[15]  Sam Kwong,et al.  Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ling Shao,et al.  Hierarchical Human Parsing With Typed Part-Relation Reasoning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.

[18]  Xuelong Li,et al.  A Review of Co-Saliency Detection Algorithms , 2018 .

[19]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Huazhu Fu,et al.  Hierarchical Features Driven Residual Learning for Depth Map Super-Resolution , 2019, IEEE Transactions on Image Processing.

[21]  Runmin Cong,et al.  Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior , 2016, IEEE Transactions on Image Processing.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Bin Du,et al.  Region-of-Interest Detection via Superpixel-to-Pixel Saliency Analysis for Remote Sensing Image , 2016, IEEE Geoscience and Remote Sensing Letters.

[24]  Qingming Huang,et al.  Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model , 2020, IEEE Transactions on Cybernetics.

[25]  Yizhou Yu,et al.  Deep Contrast Learning for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Tiantian Wang,et al.  Deep Learning for Light Field Saliency Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Sam Kwong,et al.  Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Huazhu Fu,et al.  PDR-Net: Perception-Inspired Single Image Dehazing Network With Refinement , 2020, IEEE Transactions on Multimedia.

[29]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[30]  Huchuan Lu,et al.  Salient Object Detection with Recurrent Fully Convolutional Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Huchuan Lu,et al.  Salient Object Detection via Multiple Instance Learning , 2017, IEEE Transactions on Image Processing.

[32]  Weidong Cai,et al.  Reversion Correction and Regularized Random Walk Ranking for Saliency Detection , 2018, IEEE Transactions on Image Processing.

[33]  Dewen Hu,et al.  Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast , 2015, IEEE Transactions on Image Processing.

[34]  Tian Xia,et al.  RGB-T Image Saliency Detection via Collaborative Graph Learning , 2019, IEEE Transactions on Multimedia.

[35]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[37]  Gangyi Jiang,et al.  Optimizing Multistage Discriminative Dictionaries for Blind Image Quality Assessment , 2018, IEEE Transactions on Multimedia.

[38]  Wenguan Wang,et al.  Cascaded Human-Object Interaction Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Asako Kanezaki,et al.  Salient Object Detection on Hyperspectral Images Using Features Learned from Unsupervised Segmentation Task , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Wenguan Wang,et al.  Shifting More Attention to Video Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Yang Cao,et al.  Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Dacheng Tao,et al.  An Underwater Image Enhancement Benchmark Dataset and Beyond , 2019, IEEE Transactions on Image Processing.

[43]  Ling Shao,et al.  Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement , 2015, IEEE Transactions on Image Processing.

[44]  Yanan Liu,et al.  Saliency detection based on self-adaptive multiple feature fusion for remote sensing images , 2019, International Journal of Remote Sensing.

[45]  Ling Shao,et al.  Video Salient Object Detection via Fully Convolutional Networks , 2017, IEEE Transactions on Image Processing.

[46]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[47]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[48]  Qingming Huang,et al.  Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion , 2016, IEEE Signal Processing Letters.

[49]  Peng Xu,et al.  Multigraph Transformer for Free-Hand Sketch Recognition , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Nick Barnes,et al.  Local Background Enclosure for RGB-D Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Jungong Han,et al.  RGB-T Salient Object Detection via Fusing Multi-Level CNN Features , 2019, IEEE Transactions on Image Processing.

[52]  Qingming Huang,et al.  Global Context-Aware Progressive Aggregation Network for Salient Object Detection , 2020, AAAI.

[53]  Sabine Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Tao Li,et al.  Structure-Measure: A New Way to Evaluate Foreground Maps , 2017, International Journal of Computer Vision.

[55]  Jianmin Jiang,et al.  A Simple Pooling-Based Design for Real-Time Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Kwan-Liu Ma,et al.  Stereoscopic Thumbnail Creation via Efficient Stereo Saliency Detection , 2017, IEEE Transactions on Visualization and Computer Graphics.

[57]  Peng Xu,et al.  Deep Learning for Free-Hand Sketch: A Survey , 2020, ArXiv.

[58]  Hui Xu,et al.  Co-Saliency Detection via Hierarchical Consistency Measure , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[59]  Youfu Li,et al.  Three-Stream Attention-Aware Network for RGB-D Salient Object Detection , 2019, IEEE Transactions on Image Processing.

[60]  Bing Li,et al.  Salient Object Detection via Structured Matrix Decomposition. , 2016, IEEE transactions on pattern analysis and machine intelligence.

[61]  Dewen Hu,et al.  Salient Region Detection Using Diffusion Process on a Two-Layer Sparse Graph , 2017, IEEE Transactions on Image Processing.

[62]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Fatih Murat Porikli,et al.  Underwater scene prior inspired deep underwater image and video enhancement , 2020, Pattern Recognit..

[64]  Zhiguo Jiang,et al.  Unsupervised Saliency Model with Color Markov Chain for Oil Tank Detection , 2019, Remote. Sens..

[65]  Qingming Huang,et al.  An Iterative Co-Saliency Framework for RGBD Images , 2017, IEEE Transactions on Cybernetics.

[66]  Yue Liu,et al.  Airport Extraction via Complementary Saliency Analysis and Saliency-Oriented Active Contour Model , 2018, IEEE Geoscience and Remote Sensing Letters.

[67]  Qingming Huang,et al.  ASIF-Net: Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection , 2020, IEEE Transactions on Cybernetics.

[68]  Ruigang Yang,et al.  Semi-Supervised Video Object Segmentation with Super-Trajectories , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Nianyi Li,et al.  A weighted sparse coding framework for saliency detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Guoqiang Han,et al.  R³Net: Recurrent Residual Refinement Network for Saliency Detection , 2018, IJCAI.

[71]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).