Saliency detection integrating both background and foreground information

In this paper, we propose a novel saliency detection algorithm. The saliency of an image element is defined not only as its contrast to the background but as its similarity to the foreground. First, we extract background seeds as well as their spatial layout information from image boundaries to compute the background-based saliency map. Second, we generate a compact foreground region from the first-stage saliency map to describe the appearance and location of the salient object and calculate the foreground-based saliency map accordingly. We integrate these two saliency maps and further refine the unified one to obtain a more smooth and accurate saliency map. Each component of the presented algorithm is evaluated on the public available datasets and the experimental results also show that the presented algorithm achieves favorable performance compared to the state-of-the-art methods.

[1]  Laurent Itti,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Rapid Biologically-inspired Scene Classification Using Features Shared with Visual Attention , 2022 .

[2]  Vibhav Vineet,et al.  Efficient Salient Region Detection with Soft Image Abstraction , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[4]  Sven J. Dickinson,et al.  Optimal Contour Closure by Superpixel Grouping , 2010, ECCV.

[5]  F. Qiu,et al.  Figure-ground mechanisms provide structure for selective attention , 2007, Nature Neuroscience.

[6]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  L. Itti,et al.  Quantifying center bias of observers in free viewing of dynamic natural scenes. , 2009, Journal of vision.

[8]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[9]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[10]  Vladimir Kolmogorov,et al.  Applications of parametric maxflow in computer vision , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Junjie Cao,et al.  A generalized nonlocal mean framework with object-level cues for saliency detection , 2015, The Visual Computer.

[12]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[14]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Nanning Zheng,et al.  Automatic salient object segmentation based on context and shape prior , 2011, BMVC.

[17]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[18]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[19]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[21]  Min Xu,et al.  Saliency detection with color contrast based on boundary information and neighbors , 2014, The Visual Computer.

[22]  Jian Sun,et al.  Geodesic Saliency Using Background Priors , 2012, ECCV.

[23]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Huchuan Lu,et al.  Bayesian Saliency via Low and mid Level Cues , 2022 .

[25]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[27]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[28]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Huchuan Lu,et al.  Inner and Inter Label Propagation: Salient Object Detection in the Wild , 2015, IEEE Transactions on Image Processing.

[30]  Nuno Vasconcelos,et al.  Discriminant Saliency, the Detection of Suspicious Coincidences, and Applications to Visual Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Xiaogang Wang,et al.  Saliency detection by multi-context deep learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[34]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[35]  Huchuan Lu,et al.  Saliency detection via a unified generative and discriminative model , 2016, Neurocomputing.

[36]  Huchuan Lu,et al.  Saliency detection via background and foreground seed selection , 2015, Neurocomputing.

[37]  Huchuan Lu,et al.  Deep networks for saliency detection via local estimation and global search , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.