Global contrast based salient region detection

Reliable estimation of visual saliency allows appropriate processing of images without prior knowledge of their contents, and thus remains an important step in many computer vision tasks including image segmentation, object recognition, and adaptive compression. We propose a regional contrast based saliency extraction algorithm, which simultaneously evaluates global contrast differences and spatial coherence. The proposed algorithm is simple, efficient, and yields full resolution saliency maps. Our algorithm consistently outperformed existing saliency detection methods, yielding higher precision and better recall rates, when evaluated using one of the largest publicly available data sets. We also demonstrate how the extracted saliency map can be used to create high quality segmentation masks for subsequent image processing.

[1]  H. Teuber Physiological psychology. , 1955, Annual review of psychology.

[2]  References , 1971 .

[3]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[4]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[5]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[6]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[7]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[8]  Touradj Ebrahimi,et al.  The JPEG2000 still image coding system: an overview , 2000, IEEE Trans. Consumer Electron..

[9]  S. Zeki,et al.  Inner Vision: An Exploration of Art and the Brain , 2000 .

[10]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[12]  Björn Stenger,et al.  Shape context and chamfer matching in cluttered scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[13]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[14]  R. Desimone,et al.  Interacting Roles of Attention and Visual Salience in V4 , 2003, Neuron.

[15]  P. König,et al.  Does luminance‐contrast contribute to a saliency map for overt visual attention? , 2003, The European journal of neuroscience.

[16]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[17]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[18]  Pietro Perona,et al.  Is bottom-up attention useful for object recognition? , 2004, CVPR 2004.

[19]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[20]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[21]  Byoung Chul Ko,et al.  Object-of-interest image segmentation based on human attention and semantic region clustering. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[22]  King Ngi Ngan,et al.  Unsupervised extraction of visual attention objects in color images , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[24]  Ronen Basri,et al.  Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Ralph R. Martin,et al.  Shrinkability Maps for Content‐Aware Video Resizing , 2008, Comput. Graph. Forum.

[28]  O. Sorkine,et al.  Optimized scale-and-stretch for image resizing , 2008, SIGGRAPH 2008.

[29]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[30]  Baoxin Li,et al.  A two-stage approach to saliency detection in images , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[31]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[32]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[33]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[34]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[35]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[36]  Douglas Lanman,et al.  BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields , 2009, SIGGRAPH 2009.

[37]  C. Kennard,et al.  The role of visual salience in directing eye movements in visual object agnosia , 2009, Current Biology.

[38]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[39]  Sabine Süsstrunk,et al.  Saliency detection for content-aware image resizing , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[40]  Zhuowen Tu,et al.  Shape band: A deformable object detection approach , 2009, CVPR.

[41]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[42]  Ralph R. Martin,et al.  A Shape‐Preserving Approach to Image Resizing , 2009, Comput. Graph. Forum.

[43]  Horst Bischof,et al.  Saliency driven total variation segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[44]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[45]  Hua Huang,et al.  Video Painting via Motion Layer Manipulation , 2010, Comput. Graph. Forum.

[46]  Sabine Süsstrunk,et al.  Saliency detection using maximum symmetric surround , 2010, 2010 IEEE International Conference on Image Processing.

[47]  Marc Alexa,et al.  An evaluation of descriptors for large-scale image retrieval from sketched feature lines , 2010, Comput. Graph..

[48]  Tien-Tsin Wong,et al.  Resizing by symmetry-summarization , 2010, ACM Trans. Graph..

[49]  Paria Mehrani,et al.  Saliency Segmentation based on Learning and Graph Cut Refinement , 2010, BMVC.

[50]  Shi-Min Hu,et al.  Popup: automatic paper architectures from 3D models , 2010, SIGGRAPH 2010.

[51]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, ACM Trans. Graph..

[52]  Ruofeng Tong,et al.  Content-aware copying and pasting in images , 2010, The Visual Computer.

[53]  Esa Rahtu,et al.  Segmenting Salient Objects from Images and Videos , 2010, ECCV.

[54]  Zhuowen Tu,et al.  Learning Context-Sensitive Shape Similarity by Graph Transduction , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[56]  Jianfei Cai,et al.  Content-Based Image Compression for Arbitrary-Resolution Display Devices , 2011, 2011 IEEE International Conference on Communications (ICC).

[57]  Nanning Zheng,et al.  Automatic salient object segmentation based on context and shape prior , 2011, BMVC.

[58]  Naila Murray,et al.  Saliency estimation using a non-parametric low-level vision model , 2011, CVPR 2011.

[59]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.

[60]  Hong Liu,et al.  Web-image driven best views of 3D shapes , 2011, The Visual Computer.

[61]  Yu Fu,et al.  Visual saliency detection by spatially weighted dissimilarity , 2011, CVPR 2011.

[62]  Stephen Lin,et al.  Semantic colorization with internet images , 2011, ACM Trans. Graph..

[63]  Yuzhen Niu,et al.  Rule of Thirds Detection from Photograph , 2011, 2011 IEEE International Symposium on Multimedia.

[64]  Hua Huang,et al.  Arcimboldo-like collage using internet images , 2011, ACM Trans. Graph..

[65]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2012, ECCV.

[66]  Lianhong Cai,et al.  Affective image adjustment with a single word , 2013, The Visual Computer.

[67]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Shih-Fu Chang,et al.  Mobile product search with Bag of Hash Bits and boundary reranking , 2012, CVPR.

[69]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Zhuowen Tu,et al.  Unsupervised object class discovery via saliency-guided multiple class learning , 2012, CVPR.

[71]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[72]  Olga Sorkine-Hornung,et al.  Robust Image Retargeting via Axis‐Aligned Deformation , 2012, Comput. Graph. Forum.

[73]  Shih-Fu Chang,et al.  Mobile product search with Bag of Hash Bits and boundary reranking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Shi-Min Hu,et al.  Data‐Driven Object Manipulation in Images , 2012, Comput. Graph. Forum.

[75]  Frédo Durand,et al.  A Benchmark of Computational Models of Saliency to Predict Human Fixations , 2012 .

[76]  Lihi Zelnik-Manor,et al.  Saliency for image manipulation , 2013, The Visual Computer.

[77]  Xiaochun Cao,et al.  Forgery Authentication in Extreme Wide-Angle Lens Using Distortion Cue and Fake Saliency Map , 2012, IEEE Transactions on Information Forensics and Security.

[78]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[80]  Lena Gorelick,et al.  GrabCut in One Cut , 2013, 2013 IEEE International Conference on Computer Vision.

[81]  Ming-Hsuan Yang,et al.  Discriminative Generative Contour Detection , 2013, BMVC.

[82]  Paul M. de Zeeuw,et al.  Fast saliency-aware multi-modality image fusion , 2013, Neurocomputing.

[83]  Ralph R. Martin,et al.  Internet visual media processing: a survey with graphics and vision applications , 2013, The Visual Computer.

[84]  Shi-Min Hu,et al.  Change Blindness Images , 2013, IEEE Transactions on Visualization and Computer Graphics.

[85]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[86]  Yu-Kun Lai,et al.  Artistic minimal rendering with lines and blocks , 2013, Graph. Model..

[87]  Shi-Min Hu,et al.  PoseShop: Human Image Database Construction and Personalized Content Synthesis , 2013, IEEE Transactions on Visualization and Computer Graphics.

[88]  Xuelong Li,et al.  Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search , 2013, IEEE Transactions on Image Processing.

[89]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[90]  Ali Borji,et al.  What stands out in a scene? A study of human explicit saliency judgment , 2013, Vision Research.

[91]  Jing Wang,et al.  Balance between object and background: Object-enhanced features for scene image classification , 2013, Neurocomputing.

[92]  Vibhav Vineet,et al.  Efficient Salient Region Detection with Soft Image Abstraction , 2013, 2013 IEEE International Conference on Computer Vision.

[93]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[94]  Shi-Min Hu,et al.  SalientShape: group saliency in image collections , 2013, The Visual Computer.

[95]  Yun Zhang,et al.  StereoPasting: Interactive Composition in Stereoscopic Images , 2013, IEEE Transactions on Visualization and Computer Graphics.

[96]  Ce Liu,et al.  Unsupervised Joint Object Discovery and Segmentation in Internet Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.