Saliency detection in complex scenes

AbstractDetecting multiple salient objects in complex scenes is a challenging task. In this paper, we present a novel method to detect salient objects in images. The proposed method is based on the general ‘center-surround’ visual attention mechanism and the spatial frequency response of the human visual system (HVS). The saliency computation is performed in a statistical way. This method is modeled following three biologically inspired principles and compute saliency by two ‘scatter matrices’ which are used to measure the variability within and between two classes, i.e., the center and surrounding regions, respectively. In order to detect multiple salient objects of different sizes in a scene, the saliency of a pixel is estimated via its saliency support region which is defined as the most salient region centered at the pixel. Compliance with human perceptual characteristics enables the proposed method to detect salient objects in complex scenes and predict human fixations. Experimental results on three eye tracking datasets verify the effectiveness of the method and show that the proposed method outperforms the state-of-the-art methods on the visual saliency detection task.

[1]  King Ngi Ngan,et al.  Learning to Extract Focused Objects From Low DOF Images , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[4]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[5]  Jude F. Mitchell,et al.  Spatial Attention Modulates Center-Surround Interactions in Macaque Visual Area V4 , 2009, Neuron.

[6]  D H Kelly,et al.  Motion and vision. I. Stabilized images of stationary gratings. , 1979, Journal of the Optical Society of America.

[7]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[8]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[9]  Nuno Vasconcelos,et al.  Decision-Theoretic Saliency: Computational Principles, Biological Plausibility, and Implications for Neurophysiology and Psychophysics , 2009, Neural Computation.

[10]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[11]  J. Movshon,et al.  Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[12]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[13]  Sabine Süsstrunk,et al.  Saliency detection using maximum symmetric surround , 2010, 2010 IEEE International Conference on Image Processing.

[14]  R. Shapley,et al.  Contrast's effect on spatial summation by macaque V1 neurons , 1999, Nature Neuroscience.

[15]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[17]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[18]  S. Yantis,et al.  Visual attention: control, representation, and time course. , 1997, Annual review of psychology.

[19]  Benoit M. Macq,et al.  Perceptual Image Representation , 2007, EURASIP J. Image Video Process..

[20]  Joost van de Weijer,et al.  Color Edge Saliency Boosting using Natural Image Statistics , 2010, CGIV/MCS.

[21]  Simone Frintrop,et al.  Center-surround divergence of feature statistics for salient object detection , 2011, 2011 International Conference on Computer Vision.

[22]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[23]  Wen Gao,et al.  Multi-Task Rank Learning for Visual Saliency Estimation , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[25]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[26]  Guizhong Liu,et al.  A novel attention model and its application in video analysis , 2007, Appl. Math. Comput..

[27]  Hongliang Li,et al.  Saliency detection using a central stimuli sensitivity based model , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[28]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[29]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  King Ngi Ngan,et al.  Saliency model-based face segmentation and tracking in head-and-shoulder video sequences , 2008, J. Vis. Commun. Image Represent..

[31]  HongJiang Zhang,et al.  Contrast-based image attention analysis by using fuzzy growing , 2003, MULTIMEDIA '03.

[32]  Wen Gao,et al.  Measuring visual saliency by Site Entropy Rate , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  King Ngi Ngan,et al.  A Co-Saliency Model of Image Pairs , 2011, IEEE Transactions on Image Processing.

[34]  Sabine Süsstrunk,et al.  Salient Region Detection and Segmentation , 2008, ICVS.

[35]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Xing Xie,et al.  Salient Region Detection Using Weighted Feature Maps Based on the Human Visual Attention Model , 2004, PCM.

[37]  Nuno Vasconcelos,et al.  Bottom-up saliency is a discriminant process , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[38]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[39]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Kuo-Cheng Liu Prediction error preprocessing for perceptual color image compression , 2012, EURASIP J. Image Video Process..

[42]  King Ngi Ngan,et al.  Global salient information maximization for saliency detection , 2012, Signal Process. Image Commun..

[43]  King Ngi Ngan,et al.  Object Co-Segmentation Based on Shortest Path Algorithm and Saliency Model , 2012, IEEE Transactions on Multimedia.

[44]  Argyrides Costas,et al.  Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS) , 2007 .

[45]  Ying Sun,et al.  Rigid Registration of Renal Perfusion Images Using a Neurobiology-Based Visual Saliency Model , 2010, EURASIP J. Image Video Process..

[46]  Umesh Rajashekar,et al.  DOVES: a database of visual eye movements. , 2009, Spatial vision.

[47]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[48]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[49]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[50]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[51]  King Ngi Ngan,et al.  Co-Salient Object Detection From Multiple Images , 2013, IEEE Transactions on Multimedia.

[52]  D. Heeger,et al.  When size matters: attention affects performance by contrast or response gain , 2010, Nature Neuroscience.

[53]  Henrik I. Christensen,et al.  Computational visual attention systems and their cognitive foundations: A survey , 2010, TAP.

[54]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.