How many pixels make an image?

Abstract The human visual system is remarkably tolerant to degradation in image resolution: human performance in scene categorization remains high no matter whether low-resolution images or multimegapixel images are used. This observation raises the question of how many pixels are required to form a meaningful representation of an image and identify the objects it contains. In this article, we show that very small thumbnail images at the spatial resolution of 32 × 32 color pixels provide enough information to identify the semantic category of real-world scenes. Most strikingly, this low resolution permits observers to report, with 80% accuracy, four to five of the objects that the scene contains, despite the fact that some of these objects are unrecognizable in isolation. The robustness of the information available at very low resolution for describing semantic content of natural images could be an important asset to explain the speed and efficiently at which the human brain comprehends the gist of visual scenes.

[1]  B Julesz,et al.  Masking in Visual Recognition: Effects of Two-Dimensional Filtered Noise , 1973, Science.

[2]  M. Potter Meaning in visual search. , 1975, Science.

[3]  D. Navon Forest before trees: The precedence of global features in visual perception , 1977, Cognitive Psychology.

[4]  A. Friedman Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[5]  A. Friedman Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[6]  H. Intraub Rapid conceptual identification of sequentially presented pictures. , 1981 .

[7]  T. Bachmann Identification of spatially quantised tachistoscopic images of faces: How many pixels does it take to carry identity? , 1991 .

[8]  G E Legge,et al.  Color improves object recognition in normal and low vision. , 1993, Journal of experimental psychology. Human perception and performance.

[9]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[11]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[12]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[13]  A. Oliva,et al.  Flexible, Diagnosticity-Driven, Rather Than Fixed, Perceptually Determined Scale Selection in Scene and Face Recognition , 1997, Perception.

[14]  A. Oliva,et al.  Coarse Blobs or Fine Edges? Evidence That Information Diagnosticity Changes the Perception of Complex Visual Stimuli , 1997, Cognitive Psychology.

[15]  J. Wolfe Visual memory: What do you know about what you saw? , 1998, Current Biology.

[16]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[17]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[18]  S. Klein,et al.  Measuring, estimating, and understanding the psychometric function: A commentary , 2001, Perception & psychophysics.

[19]  S. Thorpe,et al.  The Time Course of Visual Processing: From Early Perception to Decision-Making , 2001, Journal of Cognitive Neuroscience.

[20]  Ann B. Lee The Nonlinear Statistics of High-Contrast Patches in Natural Images , 2003 .

[21]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[22]  Kim Steenstrup Pedersen,et al.  The Nonlinear Statistics of High-Contrast Patches in Natural Images , 2003, International Journal of Computer Vision.

[23]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[24]  Jitendra Malik,et al.  When is scene identification just texture recognition? , 2004, Vision Research.

[25]  A. Oliva,et al.  Diagnostic Colors Contribute to the Early Stages of Scene Categorization: Behavioral and Neurophysiological Evidence , 2004 .

[26]  Abel G. Oliva,et al.  Gist of a scene , 2005 .

[27]  Olivier R. Joubert,et al.  How long to get to the “gist” of real-world natural scenes? , 2005 .

[28]  A. Oliva,et al.  Diagnostic colours contribute to the early stages of scene categorization: Behavioural and neurophysiological evidence , 2005 .

[29]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[30]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[31]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[32]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[33]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[34]  D. Field,et al.  Estimates of the information content and dimensionality of natural scenes from proximity distributions. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[35]  M. Bar The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[36]  P. Perona,et al.  What do we perceive in a glance of a real-world scene? , 2007, Journal of vision.

[37]  A. Torralba,et al.  The role of context in object recognition , 2007, Trends in Cognitive Sciences.

[38]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[39]  J. Henderson,et al.  The influence of color on the perception of scene gist. , 2008, Journal of experimental psychology. Human perception and performance.

[40]  Michelle R. Greene,et al.  Recognition of natural scenes from global properties: Seeing the forest without representing the trees , 2009, Cognitive Psychology.

[41]  Jitendra Malik,et al.  When is scene recognition just texture recognition , 2010 .