SCEGRAM: An image database for semantic and syntactic inconsistencies in scenes

Our visual environment is not random, but follows compositional rules according to what objects are usually found where. Despite the growing interest in how such semantic and syntactic rules – a scene grammar – enable effective attentional guidance and object perception, no common image database containing highly-controlled object-scene modifications has been publically available. Such a database is essential in minimizing the risk that low-level features drive high-level effects of interest, which is being discussed as possible source of controversial study results. To generate the first database of this kind – SCEGRAM – we took photographs of 62 real-world indoor scenes in six consistency conditions that contain semantic and syntactic (both mild and extreme) violations as well as their combinations. Importantly, always two scenes were paired, so that an object was semantically consistent in one scene (e.g., ketchup in kitchen) and inconsistent in the other (e.g., ketchup in bathroom). Low-level salience did not differ between object-scene conditions and was generally moderate. Additionally, SCEGRAM contains consistency ratings for every object-scene condition, as well as object-absent scenes and object-only images. Finally, a cross-validation using eye-movements replicated previous results of longer dwell times for both semantic and syntactic inconsistencies compared to consistent controls. In sum, the SCEGRAM image database is the first to contain well-controlled semantic and syntactic object-scene inconsistencies that can be used in a broad range of cognitive paradigms (e.g., verbal and pictorial priming, change detection, object identification, etc.) including paradigms addressing developmental aspects of scene grammar. SCEGRAM can be retrieved for research purposes from http://www.scenegrammarlab.com/research/scegram-database/.

[1]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[2]  R. D. Gordon,et al.  Attention to smoking-related and incongruous objects during scene viewing. , 2008, Acta psychologica.

[3]  J. Wolfe,et al.  Differential Electrophysiological Signatures of Semantic and Syntactic Scene Processing , 2013, Psychological science.

[4]  Mark W. Becker,et al.  Object-intrinsic oddities draw early saccades. , 2007, Journal of experimental psychology. Human perception and performance.

[5]  E. Wagenmakers,et al.  Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). , 2011, Journal of personality and social psychology.

[6]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[7]  J. Henderson,et al.  Does gravity matter? Effects of semantic and syntactic inconsistencies on the allocation of attention during scene perception. , 2009, Journal of vision.

[8]  Felix A. Wichmann,et al.  BOiS—Berlin Object in Scene Database: Controlled Photographic Images for Visual Search Experiments with Quantified Contextual Priors , 2016, Front. Psychol..

[9]  N. Mackworth,et al.  Cognitive determinants of fixation location during picture viewing. , 1978, Journal of experimental psychology. Human perception and performance.

[10]  J. Henderson,et al.  The effects of semantic consistency on eye movements during complex scene viewing , 1999 .

[11]  I. Biederman,et al.  Scene perception: Detecting and judging objects undergoing relational violations , 1982, Cognitive Psychology.

[12]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[13]  P. de Graef,et al.  Perceptual effects of scene context on object identification , 1990, Psychological research.

[14]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[15]  L. M. M.-T. Theory of Probability , 1929, Nature.

[16]  George L. Malcolm,et al.  Disentangling the effects of spatial inconsistency of targets and distractors when searching in realistic scenes. , 2015, Journal of vision.

[17]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[19]  B. Tatler,et al.  Semantic consistency versus perceptual salience in visual scenes: findings from change detection. , 2013, Acta psychologica.

[20]  T. Foulsham,et al.  Quarterly Journal of Experimental Psychology: in press Visual saliency and semantic incongruency influence eye movements when , 2022 .

[21]  J. Henderson,et al.  Object–scene inconsistencies do not capture gaze: evidence from the flash-preview moving-window paradigm , 2011, Attention, perception & psychophysics.

[22]  K. Rayner,et al.  Eye movements when looking at unusual/weird scenes: are there cultural differences? , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[23]  Antonio Torralba,et al.  Sharing features: efficient boosting procedures for multiclass object detection , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  G. Underwood,et al.  Congruency, saliency and gist in the inspection of objects in natural scenes , 2007 .

[25]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[26]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[27]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[28]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[29]  T. Foulsham,et al.  Is attention necessary for object identification? Evidence from eye movements during the inspection of real-world scenes , 2008, Consciousness and Cognition.

[30]  Marvin M. Chun,et al.  Babies and Brains: Habituation in Infant Cognition and Functional Neuroimaging , 2008, Frontiers in human neuroscience.