Spatiotemporal Saliency: Towards a Hierarchical Representation of Visual Saliency

In prior work, we put forth a model of visual saliency motivated by information theoretic considerations [1]. In this effort we consider how this proposal extends to explain saliency in the spatiotemporal domain and further, propose a distributed representation for visual saliency comprised of localized hierarchical saliency computation. Evidence for the efficacy of the proposal in capturing aspects of human behavior is achieved via comparison with eye tracking data and a discussion of the role of neural coding in the determination of saliency suggests avenues for future research.

[1]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[2]  P. Lennie,et al.  The Impact of Suppressive Surrounds on Chromatic Properties of Cortical Neurons , 2004, The Journal of Neuroscience.

[3]  Gabriel Kreiman,et al.  Neural coding: computational and biophysical perspectives , 2004, Physics of Life Reviews.

[4]  D M Levi,et al.  Surround modulation of perceived contrast and the role of brightness induction. , 2001, Journal of vision.

[5]  Shaul Hochstein,et al.  At first sight: A high-level pop out effect for faces , 2005, Vision Research.

[6]  Peter Földiák,et al.  SPARSE CODING IN THE PRIMATE CORTEX , 2002 .

[7]  Duje Tadin,et al.  Optimal size for perceiving motion decreases with contrast , 2005, Vision Research.

[8]  D. Sagi,et al.  Recurrent networks in human visual cortex: psychophysical evidence. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  T. W. Lee,et al.  Chromatic structure of natural scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[10]  M. Cannon,et al.  A model for inhibitory lateral interaction effects in perceived contrast , 1996, Vision Research.

[11]  R. Desimone,et al.  Spectral properties of V4 neurons in the macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  Andrew T. Smith,et al.  Surround modulation measured with functional MRI in the human visual cortex. , 2003, Journal of neurophysiology.

[13]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[14]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[15]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[16]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[17]  Robert Desimone,et al.  Top-down, but not bottom-up: Deficits in target selection in monkeys with prefrontal lesions , 2010 .

[18]  T. Albright,et al.  Adaptive Surround Modulation in Cortical Area MT , 2007, Neuron.

[19]  Ronald A. Rensink,et al.  Sensitivity To Three-Dimensional Orientation in Visual Search , 1990 .

[20]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[21]  John K. Tsotsos,et al.  An Information Theoretic Model of Saliency and Visual Search , 2008, WAPCV.

[22]  Lynn A Olzak,et al.  Contextual effects in fine spatial discriminations. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[23]  J. Gallant,et al.  Natural Stimulus Statistics Alter the Receptive Field Structure of V1 Neurons , 2004, The Journal of Neuroscience.

[24]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[25]  K. Grill-Spector,et al.  High-resolution imaging reveals highly selective nonface clusters in the fusiform face area , 2006, Nature Neuroscience.

[26]  Dennis M. Levi,et al.  Surround modulation in human vision unmasked by masking experiments , 2000, Nature Neuroscience.

[27]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[28]  H. Komatsu,et al.  Suppression on neuronal responses by a metacontrast masking stimulus in monkey V4 , 2000, Neuroscience Research.

[29]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[30]  Bin Zhang,et al.  Delayed maturation of receptive field center/surround mechanisms in V2. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[31]  R. Wurtz,et al.  Response to motion in extrastriate area MSTl: center-surround interactions. , 1998, Journal of neurophysiology.

[32]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[33]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[34]  P Cavanagh,et al.  Familiarity and pop-out in visual search , 1994, Perception & psychophysics.

[35]  S. McKee,et al.  The effect of spatial configuration on surround suppression of contrast sensitivity. , 2006, Journal of vision.

[36]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[37]  Frank E. Pollick,et al.  Obtaining features for the recognition of human movement style , 2010 .

[38]  D Sagi,et al.  The combination of spatial frequency and orientation is effortlessly perceived , 1988, Perception & psychophysics.

[39]  F. Ohl,et al.  Fallacies in behavioural interpretation of auditory cortex plasticity , 2004, Nature Reviews Neuroscience.

[40]  Chaoyi Li,et al.  Cue‐invariant detection of centre–surround discontinuity by V1 neurons in awake macaque monkey , 2007, The Journal of physiology.

[41]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[42]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[43]  V. S. Ramachandran,et al.  Perception of shape from shading , 1988, Nature.

[44]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[45]  S. Klein,et al.  Cross- and iso- oriented surrounds modulate the contrast response function: the effect of surround contrast. , 2003, Journal of vision.

[46]  D. Heeger,et al.  Center-surround interactions in foveal and peripheral vision , 2000, Vision Research.

[47]  E. Reingold,et al.  Visual search asymmetry: The influence of stimulus familiarity and low-level features , 2001, Perception & psychophysics.

[48]  D. Heeger,et al.  Measurement and modeling of center-surround suppression and enhancement , 2001, Vision Research.

[49]  John K. Tsotsos,et al.  Attention in Cognitive Systems, 5th International Workshop on Attention in Cognitive Systems, WAPCV 2008, Fira, Santorini, Greece, May 12, 2008, Revised Selected Papers , 2009, WAPCV.