A perceptual paradigm and psychophysical evidence for hierarchy in scene gist processing.

What is the order of processing in scene gist recognition? Following the seminal studies by Rosch (1978) and Tversky and Hemmenway (1983) it has been assumed that basic-level categorization is privileged over the superordinate level because the former maximizes both within-category similarity and between-category variance. However, recent research has begun to challenge this view (Oliva & Torralba, 2001; Joubert, Rousselet, Fize, & Fabre-Thorpe, 2007; Loschky & Larson, 2010). Here we study these directions more fundamentally by investigating the perceptual relations between scene categories in a way that allows us to identify the order of processing of scene categories across taxonomic levels. We introduce the category discrimination paradigm where we briefly present two real scene stimuli simultaneously and ask human observers whether they belong to the same basic-level category or not (i.e., same/different task). As we show, analysis of the obtained data reveals a hierarchical perceptual structure between different scene categories and a corresponding hierarchical structure at the perceptual processing level. In particular, we show a new type of evidence to suggest that the decision whether the scene is manmade or natural is made first, and only then more complicated decisions are taken (such as whether a manmade scene is indoor or outdoor) among a smaller set of viable candidate categories. We argue that this hierarchical structure improves performance and efficiency in both biological and artificial gist recognition systems.

[1]  Lester C. Loschky,et al.  The natural/man-made distinction is made before basic-level distinctions in scene gist processing , 2010 .

[2]  M. Potter,et al.  Recognition memory for a rapid sequence of pictures. , 1969, Journal of experimental psychology.

[3]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[4]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[5]  M. Potter.  Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[6]  I. Biederman.  Perceiving Real-World Scenes , 1972, Science.

[7]  Olivier R. Joubert,et al.  How long to get to the “gist” of real-world natural scenes? , 2005 .

[8]  Abel G. Oliva,et al.  Gist of a scene , 2005 .

[9]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[11]  I. Biederman,et al.  Searching for objects in real-world scences. , 1973, Journal of experimental psychology.

[12]  B. Tversky,et al.  Categories of environmental scenes , 1983, Cognitive Psychology.

[13]  Lester C. Loschky,et al.  Localized information is necessary for scene categorization, including the Natural/Man-made distinction. , 2008, Journal of vision.

[14]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  S. Thorpe,et al.  The time course of visual processing: Backward masking and natural scene categorisation , 2005, Vision Research.

[16]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[17]  Bernt Schiele,et al.  A Semantic Typicality Measure for Natural Scene Categorization , 2004, DAGM-Symposium.

[18]  Lester C. Loschky,et al.  The importance of information localization in scene gist recognition. , 2007, Journal of experimental psychology. Human perception and performance.

[19]  Jitendra Malik,et al.  When is scene recognition just texture recognition , 2010 .

[20]  Antonio Torralba,et al.  LabelMe: Online Image Annotation and Applications , 2010, Proceedings of the IEEE.

[21]  Mary-Ellen Large,et al.  Electrophysiological correlates of object categorization: back to basics. , 2004, Brain research. Cognitive brain research.

[22]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[23]  J. Grier,et al.  Nonparametric indexes for sensitivity and bias: computing formulas. , 1971, Psychological bulletin.

[24]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[25]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[26]  Andrew Zisserman,et al.  Scene Classification Via pLSA , 2006, ECCV.

[27]  A. Oliva,et al.  Coarse Blobs or Fine Edges? Evidence That Information Diagnosticity Changes the Perception of Complex Visual Stimuli , 1997, Cognitive Psychology.

[28]  Karl R Gegenfurtner,et al.  The dynamics of visual pattern masking in natural scene processing: a magnetoencephalography study. , 2005, Journal of vision.

[29]  Aude Oliva,et al.  Global semantic classification of scenes using power spectrum templates , 1999 .

[30]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Pietro Perona,et al.  What do we see when we glance at a scene , 2004 .

[32]  G Kovács,et al.  Cortical correlate of pattern backward masking. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[33]  P. Perona,et al.  Rapid natural scene categorization in the near absence of attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[34]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[35]  Olivier R. Joubert,et al.  The Time-Course of Visual Categorizations: You Spot the Animal Faster than the Bird , 2009, PloS one.

[36]  M. Potter.  Meaning in visual search. , 1975, Science.

[37]  T. Rogers,et al.  Object categorization: reversals and explanations of the basic-level advantage. , 2007, Journal of experimental psychology. General.

[38]  E. Rolls,et al.  The Neurophysiology of Backward Visual Masking: Information Analysis , 1999, Journal of Cognitive Neuroscience.

[39]  Antonio Torralba,et al.  Semantic Label Sharing for Learning with Many Categories , 2010, ECCV.

[40]  Jitendra Malik,et al.  When is scene identification just texture recognition? , 2004, Vision Research.

[41]  Li Fei-Fei,et al.  Neural mechanisms of rapid natural scene categorization in human visual cortex , 2009, Nature.

[42]  Guillaume A. Rousselet,et al.  Parallel processing in high-level categorization of natural images , 2002, Nature Neuroscience.

[43]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[44]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .