Visual properties and memorising scenes: Effects of image-space sparseness and uniformity

Previous studies have demonstrated that humans have a remarkable capacity to memorise a large number of scenes. The research on memorability has shown that memory performance can be predicted by the content of an image. We explored how remembering an image is affected by the image properties within the context of the reference set, including the extent to which it is different from its neighbours (image-space sparseness) and if it belongs to the same category as its neighbours (uniformity). We used a reference set of 2,048 scenes (64 categories), evaluated pairwise scene similarity using deep features from a pretrained convolutional neural network (CNN), and calculated the image-space sparseness and uniformity for each image. We ran three memory experiments, varying the memory workload with experiment length and colour/greyscale presentation. We measured the sensitivity and criterion value changes as a function of image-space sparseness and uniformity. Across all three experiments, we found separate effects of 1) sparseness on memory sensitivity, and 2) uniformity on the recognition criterion. People better remembered (and correctly rejected) images that were more separated from others. People tended to make more false alarms and fewer miss errors in images from categorically uniform portions of the image-space. We propose that both image-space properties affect human decisions when recognising images. Additionally, we found that colour presentation did not yield better memory performance over grayscale images.

[1]  C. Collin,et al.  The effects of distinctiveness on memory and metamemory for face–name associations , 2012, Memory.

[2]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[3]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[4]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[5]  James S. Nairne,et al.  Modeling Distinctiveness: Implications for General Memory Theory , 2006 .

[6]  L. Standing Learning 10000 pictures , 1973 .

[7]  M. Potter,et al.  Detecting and remembering pictures with and without visual noise. , 2008, Journal of vision.

[8]  R. Haber,et al.  Perception and memory for pictures: Single-trial learning of 2500 visual stimuli , 1970 .

[9]  Antonio Torralba,et al.  Understanding and Predicting Image Memorability at a Large Scale , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  R. VanRullen Perception Science in the Age of Deep Neural Networks , 2017, Front. Psychol..

[11]  A. Torralba,et al.  Intrinsic and extrinsic effects on image memorability , 2015, Vision Research.

[12]  Antonio Torralba,et al.  Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence , 2016, Scientific Reports.

[13]  Dimitrios Pantazis,et al.  Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks , 2015, NeuroImage.

[14]  Timothy F. Brady,et al.  Visual Long-Term Memory Has the Same Limit on Fidelity as Visual Working Memory , 2013, Psychological science.

[15]  Jeffrey S. Bowers,et al.  Detailed and gist-like visual memories are forgotten at similar rates over the course of a week , 2015, Psychonomic Bulletin & Review.

[16]  Timothy F. Brady,et al.  Conceptual Distinctiveness Supports Detailed Visual Long-term Memory for Real-world Objects the Fidelity of Long-term Memory for Visual Information , 2022 .

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  F. Craik,et al.  Depth, Elaboration, and Distinctiveness , 2014 .

[19]  Aude Oliva,et al.  Visual long-term memory has a massive storage capacity for object details , 2008, Proceedings of the National Academy of Sciences.

[20]  Gregory J Zelinsky,et al.  Effects of target typicality on categorical search. , 2014, Journal of vision.

[21]  M. Gluck,et al.  Explaining Basic Categories: Feature Predictability and Information , 1992 .

[22]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[23]  Jianxiong Xiao,et al.  What makes an image memorable? , 2011, CVPR 2011.

[24]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[25]  Timothy F. Brady,et al.  Scene Memory Is More Detailed Than You Think : The Role of Categories in Visual Long-Term Memory , 2010 .

[26]  Joel L. Voss,et al.  Long-term associative memory capacity in man , 2009, Psychonomic bulletin & review.

[27]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[28]  S. Vogt,et al.  Long-term memory for 400 pictures on a common theme. , 2007, Experimental psychology.

[29]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  Ryota Kanai,et al.  Recognising the forest, but not the trees: An effect of colour on scene perception and recognition , 2008, Consciousness and Cognition.

[32]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[33]  H Stanislaw,et al.  Calculation of signal detection theory measures , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[34]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[35]  T. Valentine The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology a Unified Account of the Effects of Distinctiveness, Inversion, and Race in Face Recognition , 2022 .

[36]  Michelle R. Greene,et al.  What you see is what you expect: rapid scene understanding benefits from prior experience , 2015, Attention, Perception, & Psychophysics.

[37]  Wilma A. Bainbridge,et al.  The intrinsic memorability of face photographs. , 2013, Journal of experimental psychology. General.