Saliency, Scale and Information: Towards a Unifying Theory

In this paper we present a definition for visual saliency grounded in information theory. This proposal is shown to relate to a variety of classic research contributions in scale-space theory, interest point detection, bilateral filtering, and to existing models of visual saliency. Based on the proposed definition of visual saliency, we demonstrate results competitive with the state-of-the art for both prediction of human fixations, and segmentation of salient objects. We also characterize different properties of this model including robustness to image transformations, and extension to a wide range of other data types with 3D mesh models serving as an example. Finally, we relate this proposal more generally to the role of saliency computation in visual information processing and draw connections to putative mechanisms for saliency computation in human vision.

[1]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[2]  Lihi Zelnik-Manor,et al.  What Makes a Patch Distinct? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[4]  Víctor Leborán,et al.  On the relationship between optical variability, visual saliency, and eye fixations: a computational approach. , 2012, Journal of vision.

[5]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jitendra Malik,et al.  Anisotropic Diffusion , 1994, Geometry-Driven Diffusion in Computer Vision.

[7]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[9]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[10]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[11]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  James M. Rehg,et al.  The Secrets of Salient Object Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Cristian Sminchisescu,et al.  CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Farzin Mokhtarian,et al.  Robust Image Corner Detection Through Curvature Scale Space , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Max A. Viergever,et al.  General intensity transformations and differential invariants , 1994, Journal of Mathematical Imaging and Vision.

[16]  John K. Tsotsos,et al.  On Sensor Bias in Experimental Methods for Comparing Interest-Point, Saliency, and Recognition Algorithms , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Martin Jägersand,et al.  Saliency Maps and Attention Selection in Scale and Spatial Coordinates: An Information Theoretic Approach , 1995, ICCV.

[18]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[19]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Bart M. ter Haar Romeny,et al.  Geometry-Driven Diffusion in Computer Vision , 1994, Computational Imaging and Vision.

[21]  Liqing Zhang,et al.  Dynamic visual attention: searching for coding length increments , 2008, NIPS.

[22]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[23]  T. Lindeberg,et al.  Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[24]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[25]  Nuno Vasconcelos,et al.  Decision-Theoretic Saliency: Computational Principles, Biological Plausibility, and Implications for Neurophysiology and Psychophysics , 2009, Neural Computation.

[26]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[27]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[28]  J. Koenderink.  The structure of images , 2004, Biological Cybernetics.

[29]  Christof Koch,et al.  Image Signature: Highlighting Sparse Salient Regions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  David W. Jacobs,et al.  Mesh saliency , 2005, SIGGRAPH 2005.

[31]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[32]  William M. Wells,et al.  A mutual-information scale-space for image feature detection and feature-based classification of volumetric brain images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.