The influence of sequential predictions on scene-gist recognition.

Past research suggests that recognizing scene gist, a viewer's holistic semantic representation of a scene acquired within a single eye fixation, involves purely feed-forward mechanisms. We investigated whether expectations can influence scene categorization. To do this, we embedded target scenes in more ecologically valid, first-person-viewpoint image sequences, along spatiotemporally connected routes (e.g., an office to a parking lot). We manipulated the sequences' spatiotemporal coherence by presenting them either coherently or in random order. Participants identified the category of one target scene in a 10-scene-image rapid serial visual presentation. Categorization accuracy was greater for targets in coherent sequences. Accuracy was also greater for targets with more visually similar primes. In Experiment 2, we investigated whether targets in coherent sequences were more predictable and whether predictable images were identified more accurately in Experiment 1 after accounting for the effect of prime-to-target visual similarity. To do this, we removed targets and had participants predict the category of the missing scene. Images were more accurately predicted in coherent sequences, and both image predictability and prime-to-target visual similarity independently contributed to performance in Experiment 1. To test whether prediction-based facilitation effects were solely due to response bias, participants performed a two-alternative forced-choice task in which they indicated whether the target was an intact or a phase-randomized scene. Critically, predictability of the target category was irrelevant to this task. Nevertheless, results showed that sensitivity, but not response bias, was greater for targets in coherent sequences. Predictions made prior to viewing a scene facilitate scene-gist recognition.

[1]  A. Agresti An introduction to categorical data analysis , 1997 .

[2]  Lester C. Loschky,et al.  Viewing Static Visual Narratives through the Lens of the Scene Perception and Event Comprehension Theory (SPECT) , 2018, Empirical Comics Research.

[3]  Michelle R. Greene,et al.  Shared Spatiotemporal Category Representations in Biological and Artificial Deep Neural Networks , 2017 .

[4]  J. Thorne,et al.  Simultaneous and preceding sounds enhance rapid visual targets: Evidence from the attentional blink , 2013, Advances in cognitive psychology.

[5]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[6]  I. Biederman,et al.  Scene perception: Detecting and judging objects undergoing relational violations , 1982, Cognitive Psychology.

[7]  M. Bar The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[8]  B. Tversky,et al.  Categories of environmental scenes , 1983, Cognitive Psychology.

[9]  Alexander Pollatsek,et al.  Extrapolating spatial layout in scene representations , 2010, Memory & cognition.

[10]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[11]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[12]  Dirk B Walther,et al.  Nonaccidental Properties Underlie Human Categorization of Complex Natural Scenes , 2014, Psychological science.

[13]  Nathalie Guyader,et al.  Image phase or amplitude? Rapid scene categorization is an amplitude-based process. , 2004, Comptes rendus biologies.

[14]  Michael Bach,et al.  The Freiburg Visual Acuity Test-Variability unchanged by post-hoc re-analysis , 2007, Graefe's Archive for Clinical and Experimental Ophthalmology.

[15]  E. Williams Experimental Designs Balanced for the Estimation of Residual Effects of Treatments , 1949 .

[16]  Jodi L. Davenport,et al.  Scene Consistency in Object and Background Perception , 2004, Psychological science.

[17]  Brian J. Scholl,et al.  Enhanced visual awareness for morality and pajamas? Perception vs. memory in ‘top-down’ effects , 2015, Cognition.

[18]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[19]  Michelle R. Greene,et al.  What you see is what you expect: rapid scene understanding benefits from prior experience , 2015, Attention, Perception, & Psychophysics.

[20]  Thomas Sanocki Representation and perception of spatial layout , 2003 .

[21]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[22]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[23]  Dirk B. Walther,et al.  Contour junctions underlie neural representations of scene categories in high-level human visual cortex , 2016 .

[24]  Kara D. Federmeier,et al.  The N400 as a snapshot of interactive processing: Evidence from regression analyses of orthographic neighbor and lexical associate effects. , 2011, Psychophysiology.

[25]  Jeffrey M. Zacks,et al.  Event perception: a mind-brain perspective. , 2007, Psychological bulletin.

[26]  A. Raftery Bayesian Model Selection in Social Research , 1995 .

[27]  W. F. Hawkins,et al.  Vexierversuch: the log relationship between word-frequency and recognition obtained in the absence of stimulus words. , 1958, Journal of experimental psychology.

[28]  P A Salin,et al.  Corticocortical connections in the visual system: structure and function. , 1995, Physiological reviews.

[29]  C Kaernbach,et al.  A single-interval adjustment-matrix (SIAM) procedure for unbiased adaptive testing. , 1990, The Journal of the Acoustical Society of America.

[30]  Lester C. Loschky,et al.  The contribution of amplitude and phase spectra-defined scene statistics to the masking of rapid scene categorization. , 2013, Journal of vision.

[31]  Lucy S. Petro,et al.  Contextual Feedback to Superficial Layers of V1 , 2015, Current Biology.

[32]  M. Kutas,et al.  An electrophysiological study of scene effects on object identification. , 2003, Brain research. Cognitive brain research.

[33]  S. Ullman,et al.  Spatial Context in Recognition , 1996, Perception.

[34]  K. Rayner The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search , 2009, Quarterly journal of experimental psychology.

[35]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[36]  E. Wagenmakers,et al.  AIC model selection using Akaike weights , 2004, Psychonomic bulletin & review.

[37]  R. Sperber,et al.  Semantic priming effects on picture and word processing , 1979 .

[38]  Antonio Torralba,et al.  Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence , 2016, Scientific Reports.

[39]  Ryan V. Ringer,et al.  The spatiotemporal dynamics of scene gist recognition. , 2014, Journal of experimental psychology. Human perception and performance.

[40]  Blake W. Johnson,et al.  Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts , 2002, Clinical Neurophysiology.

[41]  Marius V. Peelen,et al.  Interaction between Scene and Object Processing Revealed by Human fMRI and MEG Decoding , 2017, The Journal of Neuroscience.

[42]  L. Deouell,et al.  ERP evidence for context congruity effects during simultaneous object–scene processing , 2010, Neuropsychologia.

[43]  J. Henderson,et al.  Does consistent scene context facilitate object perception? , 1998, Journal of experimental psychology. General.

[44]  Olivier R. Joubert,et al.  How long to get to the “gist” of real-world natural scenes? , 2005 .

[45]  Guillaume A. Rousselet,et al.  Rapid visual categorization of natural scene contexts with equalized amplitude spectrum and increasing phase noise. , 2009, Journal of vision.

[46]  S. Thorpe,et al.  Surfing a spike wave down the ventral stream , 2002, Vision Research.

[47]  S. Thorpe,et al.  A Limit to the Speed of Processing in Ultra-Rapid Visual Categorization of Novel Natural Scenes , 2001, Journal of Cognitive Neuroscience.

[48]  S. Thorpe,et al.  The time course of visual processing: Backward masking and natural scene categorisation , 2005, Vision Research.

[49]  T. Jaeger,et al.  Categorical Data Analysis: Away from ANOVAs (transformation or not) and towards Logit Mixed Models. , 2008, Journal of memory and language.

[50]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[51]  B. Scholl,et al.  Cognition does not affect perception: Evaluating the evidence for “top-down” effects , 2015, Behavioral and Brain Sciences.

[52]  M. Potter Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[53]  Liad Mudrik,et al.  Are incongruent objects harder to identify? The functional significance of the N300 component , 2018, Neuropsychologia.

[54]  W. Epstein,et al.  Priming Spatial Layout of Scenes , 1997 .

[55]  Bruce C Hansen,et al.  The role of higher order image statistics in masking scene gist recognition , 2010, Attention, perception & psychophysics.

[56]  L. T. DeCarlo Signal detection theory and generalized linear models , 1998 .

[57]  Kathy Pezdek,et al.  Memory for Real-World Scenes: The Role of Consistency With Schema Expectation , 1989 .

[58]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[59]  J. Wolfe,et al.  Differential Electrophysiological Signatures of Semantic and Syntactic Scene Processing , 2013, Psychological science.

[60]  M. Bar,et al.  Early onset of neural synchronization in the contextual associations network , 2011, Proceedings of the National Academy of Sciences.

[61]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[62]  I. Biederman,et al.  On the information extracted from a glance at a scene. , 1974, Journal of experimental psychology.

[63]  Bruce C Hansen,et al.  Structural sparseness and spatial phase alignment in natural scenes. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[64]  Lester C. Loschky,et al.  Visual information representation and rapid-scene categorization are simultaneous across cortex: An MEG study , 2016, NeuroImage.

[65]  Lester C. Loschky,et al.  The importance of information localization in scene gist recognition. , 2007, Journal of experimental psychology. Human perception and performance.

[66]  Lester C. Loschky,et al.  Localized information is necessary for scene categorization, including the Natural/Man-made distinction. , 2008, Journal of vision.

[67]  Li Fei-Fei,et al.  Categorization influences detection: A perceptual advantage for representative exemplars of natural scene categories , 2017, Journal of vision.

[68]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[69]  E. Wright,et al.  Effects of semantic priming on visual encoding of pictures. , 1989, Journal of experimental psychology. General.

[70]  S Ullman,et al.  Sequence seeking and counter streams: a computational model for bidirectional information flow in the visual cortex. , 1995, Cerebral cortex.

[71]  Daniel B Wright,et al.  Functions for traditional and multilevel approaches to signal detection theory , 2009, Behavior research methods.