Combining multi-layer integration algorithm with background prior and label propagation for saliency detection

Abstract In this paper, we propose a novel approach to automatically detect salient regions in an image. Firstly, some corner superpixels serve as the background labels and the saliency of other superpixels are determined by ranking their similarities to the background labels based on ranking algorithm. Subsequently, we further employ an objectness measure to pick out and propagate foreground labels. Furthermore, an integration algorithm is devised to fuse both background-based saliency map and foreground-based saliency map, meanwhile an original energy function is acted as refinement before integration. Finally, results from multiscale saliency maps are integrated to further improve the detection performance. Our experimental results on five benchmark datasets demonstrate the effectiveness of the proposed method. Our method produces more accurate saliency maps with better precision-recall curve, higher F-measure and lower mean absolute error than other 13 state-of-the-arts approaches on ASD, SED, ECSSD, iCoSeg and PASCAL-S datasets.

[1]  Liang Lin,et al.  PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures with Spatial Priors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Chun Chen,et al.  Low-level and high-level prior learning for visual saliency estimation , 2014, Inf. Sci..

[4]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[5]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[7]  Radomír Mech,et al.  Minimum Barrier Salient Object Detection at 80 FPS , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  C. Lawrence Zitnick,et al.  Structured Forests for Fast Edge Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  David Dagan Feng,et al.  Robust saliency detection via regularized random walks ranking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Garrison W. Cottrell,et al.  Robust classification of objects, faces, and flowers using natural image statistics , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Shang-Hong Lai,et al.  Fusing generic objectness and visual saliency for salient object detection , 2011, 2011 International Conference on Computer Vision.

[13]  Jie Yang,et al.  Normalized cut-based saliency detection by adaptive multi-level region merging , 2015, IEEE Transactions on Image Processing.

[14]  Nuno Vasconcelos,et al.  Learning Optimal Seeds for Diffusion-Based Salient Object Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[16]  Chun Chen,et al.  What Is the Chance of Happening: A New Way to Predict Where People Look , 2010, ECCV.

[17]  Huchuan Lu,et al.  Visual saliency detection based on Bayesian model , 2011, 2011 18th IEEE International Conference on Image Processing.

[18]  Huchuan Lu,et al.  Saliency Detection with Multi-Scale Superpixels , 2014, IEEE Signal Processing Letters.

[19]  Xingming Sun,et al.  Effective and Efficient Global Context Verification for Image Copy Detection , 2017, IEEE Transactions on Information Forensics and Security.

[20]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[21]  David A. Clausi,et al.  Statistical Textural Distinctiveness for Salient Region Detection in Natural Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Mei Han,et al.  Category-Independent Object-Level Saliency Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Feng Wu,et al.  Background Prior-Based Salient Object Detection via Deep Reconstruction Residual , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[25]  Min Xu,et al.  Saliency detection with color contrast based on boundary information and neighbors , 2014, The Visual Computer.

[26]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Nanning Zheng,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[28]  James M. Rehg,et al.  The Secrets of Salient Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Huchuan Lu,et al.  Graph-Regularized Saliency Detection With Convex-Hull-Based Center Prior , 2013, IEEE Signal Processing Letters.

[30]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Huchuan Lu,et al.  Saliency Detection via Absorbing Markov Chain , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[33]  Yuan Yan Tang,et al.  High-Order Distance-Based Multiview Stochastic Learning in Image Classification , 2014, IEEE Transactions on Cybernetics.

[34]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  B. Schölkopf,et al.  Graph-Based Visual Saliency , 2007 .

[36]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Huchuan Lu,et al.  Saliency detection via Cellular Automata , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Fei Gao,et al.  Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking , 2017, IEEE Transactions on Cybernetics.

[39]  Dongqing Zhang,et al.  Color filter for image search , 2012, ACM Multimedia.

[40]  Hanling Zhang,et al.  Saliency Detection Combining Multi-layer Integration Algorithm with Background Prior and Energy Function , 2016, PCM.

[41]  Jiebo Luo,et al.  Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance , 2011, International Journal of Computer Vision.

[42]  Aykut Erdem,et al.  Visual saliency estimation by nonlinearly integrating features using region covariances. , 2013, Journal of vision.

[43]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Huchuan Lu,et al.  Inner and Inter Label Propagation: Salient Object Detection in the Wild , 2015, IEEE Transactions on Image Processing.

[45]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[46]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[47]  Peng Jiang,et al.  Salient Region Detection by UFO: Uniqueness, Focusness and Objectness , 2013, 2013 IEEE International Conference on Computer Vision.

[48]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Matthew H Tong,et al.  SUN: Top-down saliency using natural statistics , 2009, Visual cognition.

[50]  Gabriela Csurka,et al.  A framework for visual saliency detection with applications to image thumbnailing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[51]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Huchuan Lu,et al.  Salient object detection via bootstrap learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Jianping Fan,et al.  iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning , 2017, IEEE Transactions on Information Forensics and Security.

[54]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Yu Fu,et al.  Visual saliency detection by spatially weighted dissimilarity , 2011, CVPR 2011.

[56]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Esa Rahtu,et al.  Fast and Efficient Saliency Detection Using Sparse Sampling and Kernel Density Estimation , 2011, SCIA.

[58]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[59]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.