Benchmark and Evaluation Metrics

This Chapter reviews the benchmarks and evaluation metrics for visual saliency computation. It consists of four main parts. The first part provides an overview of saliency model evaluation. In the second part, we review existing saliency benchmarks. Based on these benchmarks, the third part describes various evaluation metrics for measuring the performance of saliency models. The last part concludes this Chapter and proposes how to achieve impressive performance when using these benchmarks and evaluation methodologies.

[1]  Pierre Baldi,et al.  A principled approach to detecting surprising events in video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Nathalie Guyader,et al.  Modelling Spatio-Temporal Saliency to Predict Gaze Direction for Short Videos , 2009, International Journal of Computer Vision.

[3]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[4]  Nuno Vasconcelos,et al.  Discriminant Saliency, the Detection of Suspicious Coincidences, and Applications to Visual Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  L. Itti Author address: , 1999 .

[6]  Christof Koch,et al.  Learning a saliency map using fixated locations in natural scenes. , 2011, Journal of vision.

[7]  Liqing Zhang,et al.  Dynamic visual attention: searching for coding length increments , 2008, NIPS.

[8]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[9]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[10]  Laurent Itti,et al.  Congruence between model and human attention reveals unique signatures of critical visual events , 2007, NIPS.

[11]  Zhiwen Yu,et al.  A Rule Based Technique for Extraction of Visual Attention Regions Based on Real-Time Clustering , 2007, IEEE Transactions on Multimedia.

[12]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[13]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[14]  M. Land Eye movements and the control of actions in everyday life , 2006, Progress in Retinal and Eye Research.

[15]  Bernard Chalmond,et al.  Using hidden scale for salient object detection , 2006, IEEE Transactions on Image Processing.

[16]  King Ngi Ngan,et al.  Towards unsupervised attention object extraction by integrating visual attention and object growing , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[17]  Yung-Yu Chuang,et al.  A collaborative benchmark for region of interest detection algorithms , 2009, CVPR.

[18]  Giulio Sandini,et al.  Object-based Visual Attention: a Model for a Behaving Robot , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[19]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[20]  Wen-Huang Cheng,et al.  Automatic video region-of-interest determination based on user attention model , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[21]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Nuno Vasconcelos,et al.  The discriminant center-surround hypothesis for bottom-up saliency , 2007, NIPS.

[23]  Laurent Itti,et al.  The role of memory in guiding attention during natural vision. , 2006, Journal of vision.

[24]  Nanning Zheng,et al.  Video attention: Learning to detect a salient object sequence , 2008, 2008 19th International Conference on Pattern Recognition.

[25]  Shan Li,et al.  Efficient spatiotemporal-attention-driven shot matching , 2007, ACM Multimedia.

[26]  Bernhard Schölkopf,et al.  A Nonparametric Approach to Bottom-Up Visual Saliency , 2006, NIPS.

[27]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[28]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[30]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Yannis Avrithis,et al.  Bottom-up spatiotemporal visual attention model for video analysis , 2007 .

[32]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[33]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[34]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[35]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Liang-Tien Chia,et al.  Adaptive local context suppression of multiple cues for salient visual attention detection , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[37]  Christof Koch,et al.  Feature combination strategies for saliency-based visual attention systems , 2001, J. Electronic Imaging.

[38]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[39]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[40]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Ki Tae Park,et al.  Automatic Extraction of Salient Objects using Feature Maps , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[42]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2012, ECCV.

[43]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[44]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[45]  Wen Gao,et al.  A dataset and evaluation methodology for visual saliency in video , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[46]  Wen Gao,et al.  Measuring visual saliency by Site Entropy Rate , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Laurent Itti,et al.  Applying computational tools to predict gaze direction in interactive visual environments , 2008, TAP.

[48]  Thomas Deselaers,et al.  ClassCut for Unsupervised Class Segmentation , 2010, ECCV.

[49]  Laurent Itti,et al.  Beyond bottom-up: Incorporating task-dependent influences into a computational model of spatial attention , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Christof Koch,et al.  Predicting human gaze using low-level saliency combined with face detection , 2007, NIPS.

[51]  Brian Scassellati,et al.  A Behavioral Analysis of Computational Models of Visual Attention , 2007, International Journal of Computer Vision.

[52]  Deepu Rajan,et al.  Random walks on graphs to model saliency in images , 2009, CVPR.

[53]  Nuno Vasconcelos,et al.  On the plausibility of the discriminant center-surround hypothesis for visual saliency. , 2008, Journal of vision.

[54]  P. König,et al.  Differences of monkey and human overt attention under natural conditions , 2006, Vision Research.

[55]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[56]  Kunio Kashino,et al.  A stochastic model of selective visual attention with a dynamic Bayesian network , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[57]  Nanning Zheng,et al.  Learning to Detect A Salient Object , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Tiejun Huang,et al.  Visual Saliency with Statistical Priors , 2013, International Journal of Computer Vision.

[59]  Tsuhan Chen,et al.  Determining Patch Saliency Using Low-Level Context , 2008, ECCV.

[60]  Harish Katti,et al.  An Eye Fixation Database for Saliency Detection in Images , 2010, ECCV.