Deep learning framework for saliency object detection based on global prior and local context

Abstract. The saliency object detection is a hot topic of computer vision. Traditional saliency detection methods are overly dependent on handcrafted low-level features. The saliency detection methods based on deep learning can effectively solve the problem, which extracts high-level features automatically. However, there are some noises in the extracted high-level features that affect the detection performance. We propose a deep learning framework for saliency detection based on global prior and local context. First, we use feature maps generated by combining some middle-level features as the input of global-prior-based deep learning model, which can reduce the interference of distracting feature information for the saliency detection. Then, two deep learning models use respectively local contexts of color image and depth map as input, which combine global prior to generate the initial saliency map. Finally, the optimized saliency map can be obtained based on spatial consistence and appearance similarity. Experiments on two publicly available datasets show that the proposed method performs better than other nine state-of-the-art approaches.

[1]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ioannis Rigas,et al.  Efficient modeling of visual saliency based on local sparse representation and the use of hamming distance , 2015, Comput. Vis. Image Underst..

[3]  Michael Ying Yang,et al.  Exploiting global priors for RGB-D saliency detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Ping Hu,et al.  Detecting Salient Objects via Color and Texture Compactness Hypotheses. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[5]  Huchuan Lu,et al.  Learning Uncertain Convolutional Features for Accurate Saliency Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Leon A. Gatys,et al.  Understanding Low- and High-Level Contributions to Fixation Prediction , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Yang Liu,et al.  Depth-aware salient object detection using anisotropic center-surround difference , 2015, Signal Process. Image Commun..

[8]  Yizhou Yu,et al.  Visual saliency based on multiscale deep features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Huchuan Lu,et al.  Salient object detection via global and local cues , 2015, Pattern Recognit..

[10]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[11]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Min Xu,et al.  Saliency detection with color contrast based on boundary information and neighbors , 2014, The Visual Computer.

[13]  Jin Liu,et al.  Saliency Pattern Detection by Ranking Structured Trees , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Jianmin Wang,et al.  Deep Visual-Semantic Quantization for Efficient Image Retrieval , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Chu-Song Chen,et al.  Salient object detection via local saliency estimation and global homogeneity refinement , 2014, Pattern Recognit..

[18]  Xuelong Li,et al.  Spatiochromatic Context Modeling for Color Saliency Analysis , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Jiandong Tian,et al.  RGBD Salient Object Detection via Deep Fusion , 2016, IEEE Transactions on Image Processing.

[20]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[22]  Qingming Huang,et al.  Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion , 2016, IEEE Signal Processing Letters.

[23]  Dewen Hu,et al.  Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast , 2015, IEEE Transactions on Image Processing.

[24]  Jin Young Choi,et al.  Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Yizhou Yu,et al.  Deep Contrast Learning for Salient Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Jan P. Allebach,et al.  Model-Based Iterative Restoration for Binary Document Image Compression with Dictionary Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  King Ngi Ngan,et al.  Global salient information maximization for saliency detection , 2012, Signal Process. Image Commun..

[30]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[31]  Liaoyuan Zeng,et al.  An effective vector model for global-contrast-based saliency detection , 2015, J. Vis. Commun. Image Represent..

[32]  Zhuowen Tu,et al.  Deeply Supervised Salient Object Detection with Short Connections , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).