Visual objects in context

We see the world in scenes, where visual objects occur in rich surroundings, often embedded in a typical context with other related objects. How does the human brain analyse and use these common associations? This article reviews the knowledge that is available, proposes specific mechanisms for the contextual facilitation of object recognition, and highlights important open questions. Although much has already been revealed about the cognitive and cortical mechanisms that subserve recognition of individual objects, surprisingly little is known about the neural underpinnings of contextual analysis and scene perception. Building on previous findings, we now have the means to address the question of how the brain integrates individual elements to construct the visual experience.

[1]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .

[2]  J. Piaget The child's construction of reality , 1954 .

[3]  R. T. Green,et al.  Information theory and figure perception: the metaphor that failed. , 1966, Acta Psychologica.

[4]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[5]  I. Biederman,et al.  On the information extracted from a glance at a scene. , 1974, Journal of experimental psychology.

[6]  R. Hirsh The hippocampus and contextual retrieval of information from memory: a theory. , 1974, Behavioral biology.

[7]  M. Potter,et al.  Time to understand pictures and words , 1975, Nature.

[8]  Ronald C. Shank,et al.  Theoretical Issues in Natural Language Processing , 1975 .

[9]  tephen E. Palmer The effects of contextual scenes on the identification of objects , 1975, Memory & cognition.

[10]  Roger C. Schank,et al.  Using Knowledge to Understand , 1975, TINLAP.

[11]  Patrick Henry Winston,et al.  The psychology of computer vision , 1976, Pattern Recognit..

[12]  J M Mandler,et al.  Some of the thousand words a picture is worth. , 1976, Journal of experimental psychology. Human learning and memory.

[13]  J. Senders,et al.  Eye Movements and Psychological Processes , 1976 .

[14]  M. Potter Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[15]  J. Antes Recognizing and localizing features in brief picture presentations , 1977, Memory & cognition.

[16]  D. Taylor Eye Movements and Psychological Processes , 1977 .

[17]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[18]  Howard S. Hock,et al.  Real-world schemata and scene recognition in adults and children , 1978 .

[19]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[20]  C. Richard Puff,et al.  Memory Organization and Structure , 1979 .

[21]  A. Friedman Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[22]  A. Friedman Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[23]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[24]  T H Carr,et al.  Early extraction of meaning from pictures and its relation to conscious identification. , 1980, Journal of experimental psychology. Human perception and performance.

[25]  M. Eckardt The Hippocampus as a Cognitive Map , 1980 .

[26]  M. C. Smith,et al.  Tracing the time course of picture--word processing. , 1980, Journal of experimental psychology. General.

[27]  S. Grossberg,et al.  How does a brain build a cognitive code? , 1980, Psychological review.

[28]  H. Intraub Rapid conceptual identification of sequentially presented pictures. , 1981 .

[29]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[30]  Robert M. Schindler,et al.  Error in proofreading: Evidence of syntactic control of letter processing? , 1981 .

[31]  T. Carr,et al.  Words, pictures, and priming: on semantic activation, conscious identification, and the automaticity of information processing. , 1982, Journal of experimental psychology. Human perception and performance.

[32]  I. Biederman,et al.  Scene perception: Detecting and judging objects undergoing relational violations , 1982, Cognitive Psychology.

[33]  Richard L. Metzger,et al.  The nature of processing early in picture perception , 1983, Psychological research.

[34]  S Abramovici,et al.  Errors in proofreading: Evidence for syntactic control of letter processing? , 1983, Memory & cognition.

[35]  H. Intraub Conceptual masking: the effects of subsequent visual events on memory for pictures. , 1984, Journal of experimental psychology. Learning, memory, and cognition.

[36]  W. Glaser,et al.  The time course of picture-word interference. , 1984 .

[37]  E. Halgren,et al.  Human medial temporal lobe potentials evoked in memory and language tasks. , 1986, Electroencephalography and clinical neurophysiology.

[38]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[39]  D. Amaral,et al.  The entorhinal cortex of the monkey: II. Cortical afferents , 1987, The Journal of comparative neurology.

[40]  N. Kanwisher Repetition blindness: Type recognition without token individuation , 1987, Cognition.

[41]  E Valenstein,et al.  Retrosplenial amnesia. , 1987, Brain : a journal of neurology.

[42]  Glyn W. Humphreys,et al.  Semantic systems or system? Neuropsychological evidence re-examined , 1988 .

[43]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[44]  S. Ullman Aligning pictorial descriptions: An approach to object recognition , 1989, Cognition.

[45]  K. Rayner,et al.  The effect of background information on object identification , 2017 .

[46]  G. Murphy,et al.  Categorizing objects in isolation and in scenes: what a superordinate is good for. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[47]  Alan S. Brown,et al.  Cryptomnesia: Delineating inadvertent plagiarism. , 1989 .

[48]  G. Davies,et al.  Memory in context : context in memory , 1990 .

[49]  Dr. Robert Miller Cortico-Hippocampal Interplay and the Representation of Contexts in the Brain , 1991, Studies of Brain Function.

[50]  R. Jaffard [Neurobiology of memory]. , 1991, La Revue du praticien.

[51]  A. Paivio Dual coding theory: Retrospect and current status. , 1991 .

[52]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[53]  W. Glaser Picture naming , 1992, Cognition.

[54]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[55]  P. de Graef,et al.  Local and global contextual constraints on the identification of objects in scenes. , 1992, Canadian journal of psychology.

[56]  Adrienne Lehrer,et al.  Frames, Fields, and Contrasts : New Essays in Semantic and Lexical Organization , 1992 .

[57]  John H. R. Maunsell,et al.  How parallel are the primate visual pathways? , 1993, Annual review of neuroscience.

[58]  K Tanaka,et al.  Neuronal mechanisms of object recognition. , 1993, Science.

[59]  P. Holcomb,et al.  Event-Related Brain Potentials Reflect Semantic Priming in an Object Decision Task , 1994, Brain and Cognition.

[60]  S. Kosslyn Image and Brain , 1994 .

[61]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[62]  E. Halgren,et al.  Spatio-temporal stages in face and word processing. 2. Depth-recorded potentials in the human frontal and Rolandic cortices , 1994, Journal of Physiology-Paris.

[63]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[64]  D. Gaffan Scene-Specific Memory for Objects: A Model of Episodic Memory Impairment in Monkeys with Fornix Transection , 1994, Journal of Cognitive Neuroscience.

[65]  S. Kosslyn Image and Brain: The Resolution of the Imagery Debate , 1994, Journal of Cognitive Neuroscience.

[66]  S Ullman,et al.  Sequence seeking and counter streams: a computational model for bidirectional information flow in the visual cortex. , 1995, Cerebral cortex.

[67]  G. McCarthy,et al.  Language-related field potentials in the anterior-medial temporal lobe: I. Intracranial distribution and neural generators , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[68]  J. Bullier,et al.  Parallel versus serial processing: new vistas on the distributed organization of the visual system , 1995, Current Opinion in Neurobiology.

[69]  Richard S. J. Frackowiak,et al.  Functional anatomy of a common semantic system for words and pictures , 1996, Nature.

[70]  H. Intraub,et al.  Boundary Extension for Briefly Glimpsed Photographs: Do Common Perceptual Processes Result in Unexpected Memory Distortions? , 1996 .

[71]  S. Ullman,et al.  Spatial Context in Recognition , 1996, Perception.

[72]  M. D’Esposito,et al.  The parahippocampus subserves topographical learning in man , 1996, NeuroImage.

[73]  J. R. Baker,et al.  The hippocampal formation participates in novel picture encoding: evidence from functional magnetic resonance imaging. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[74]  Jeremy M. Wolfe,et al.  Just Say No: How Are Visual Searches Terminated When There Is No Target Present? , 1996, Cognitive Psychology.

[75]  Leslie G. Ungerleider,et al.  Neural correlates of category-specific knowledge , 1996, Nature.

[76]  Y. Miyashita,et al.  Formation of mnemonic neuronal responses to visual paired associates in inferotemporal cortex is impaired by perirhinal and entorhinal lesions. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[77]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[78]  A. Oliva,et al.  Flexible, Diagnosticity-Driven, Rather Than Fixed, Perceptually Determined Scale Selection in Scene and Face Recognition , 1997, Perception.

[79]  H. Intraub The representation of visual scenes , 1997, Trends in Cognitive Sciences.

[80]  D. S. Wooding,et al.  Fixation Patterns Made during Brief Examination of Two-Dimensional Images , 1997, Perception.

[81]  W. Epstein,et al.  Priming Spatial Layout of Scenes , 1997 .

[82]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[83]  I. Biederman,et al.  Subliminal visual priming , 1997, Neuroscience Letters.

[84]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[85]  Jean Bullier,et al.  The Timing of Information Transfer in the Visual System , 1997 .

[86]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[87]  R. Morris,et al.  Contingent versus incidental context processing during conditioning: Dissociation after excitotoxic hippocampal plus dentate gyrus lesions , 1998, Hippocampus.

[88]  E. Maguire,et al.  Knowing Where Things Are: Parahippocampal Involvement in Encoding Object Locations in Virtual Large-Scale Space , 1998, Journal of Cognitive Neuroscience.

[89]  Michael S. Gazzaniga,et al.  Creating false memories for visual scenes , 1998, Neuropsychologia.

[90]  D. Gaffan,et al.  Perirhinal cortex ablation impairs configural learning and paired–associate learning equally , 1998, Neuropsychologia.

[91]  J. Desmond,et al.  The role of left prefrontal cortex in language and memory. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[92]  J. Henderson,et al.  Does consistent scene context facilitate object perception? , 1998, Journal of experimental psychology. General.

[93]  M. Chun,et al.  Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention , 1998, Cognitive Psychology.

[94]  M. Chun,et al.  Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage , 1999, Nature Neuroscience.

[95]  R. G. Morris D.O. Hebb: The Organization of Behavior, Wiley: New York; 1949 , 1999, Brain Research Bulletin.

[96]  Kenji Kawano,et al.  Global and fine information coded by single neurons in the temporal visual cortex , 1999, Nature.

[97]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[98]  D. Schacter,et al.  Medial temporal lobe activations in fMRI and PET studies of episodic encoding and retrieval , 1999, Hippocampus.

[99]  C V Gottesman,et al.  Wide-angle memories of close-up scenes: A demonstration of boundary extension , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[100]  H. Bülthoff,et al.  View dependence in scene recognition after active learning , 1999, Memory & cognition.

[101]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[102]  H G Wieser,et al.  Human hippocampus associates information in memory. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[103]  E. Halgren,et al.  Dynamic Statistical Parametric Mapping Combining fMRI and MEG for High-Resolution Imaging of Cortical Activity , 2000, Neuron.

[104]  P. Holcomb,et al.  Imaginal, Semantic, and Surface-Level Processing of Concrete and Abstract Words: An Electrophysiological Investigation , 2000, Journal of Cognitive Neuroscience.

[105]  T. Schormann,et al.  Functional delineation of the human occipito-temporal areas related to face and scene processing. A PET study. , 2000, Brain : a journal of neurology.

[106]  H. Barbas,et al.  The laminar pattern of connections between prefrontal and anterior temporal cortices in the Rhesus monkey is related to cortical structure and function. , 2000, Cerebral cortex.

[107]  M. Chun,et al.  Contextual cueing of visual attention , 2022 .

[108]  Y. Miyashita,et al.  Backward spreading of memory-retrieval signal in the primate temporal cortex. , 2001, Science.

[109]  M. Bar,et al.  Cortical Mechanisms Specific to Explicit Visual Object Recognition , 2001, Neuron.

[110]  C. Gilbert,et al.  On a common circle: natural scenes and Gestalt rules. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[111]  Talma Hendler,et al.  Center–periphery organization of human object areas , 2001, Nature Neuroscience.

[112]  S. Mizumori,et al.  Temporary Inactivation of the Retrosplenial Cortex Causes a Transient Reorganization of Spatial Coding in the Hippocampus , 2001, The Journal of Neuroscience.

[113]  N. Kanwisher,et al.  The Human Body , 2001 .

[114]  H. Tamura,et al.  Visual response properties of cells in the ventral and dorsal parts of the macaque inferotemporal cortex. , 2001, Cerebral cortex.

[115]  Kara D. Federmeier,et al.  Meaning and modality: influences of context, semantic memory organization, and perceptual predictability on picture processing. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[116]  L. Squire,et al.  Simple and associative recognition memory in the hippocampal region. , 2001, Learning & memory.

[117]  E. Maguire The retrosplenial contribution to human navigation: a review of lesion and neuroimaging findings. , 2001, Scandinavian journal of psychology.

[118]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[119]  H. Eichenbaum The hippocampus and declarative memory: cognitive mechanisms and neural codes , 2001, Behavioural Brain Research.

[120]  T Allison,et al.  Contextual guidance of attention: human intracranial event-related potential evidence for feedback modulation in anatomically early temporally late stages of visual processing. , 2001, Brain : a journal of neurology.

[121]  A. David Redish,et al.  The hippocampal debate: are we asking the right questions? , 2001, Behavioural Brain Research.

[122]  Malcolm W. Brown,et al.  Recognition memory: What are the roles of the perirhinal cortex and hippocampus? , 2001, Nature Reviews Neuroscience.

[123]  N. Kanwisher,et al.  The lateral occipital complex and its role in object recognition , 2001, Vision Research.

[124]  E. Maguire,et al.  A Temporoparietal and Prefrontal Network for Retrieving the Spatial Context of Lifelike Events , 2001, NeuroImage.

[125]  M. Rugg,et al.  Context Effects on the Neural Correlates of Recognition Memory An Electrophysiological Study , 2001, Neuron.

[126]  S. Thorpe,et al.  The Time Course of Visual Processing: From Early Perception to Decision-Making , 2001, Journal of Cognitive Neuroscience.

[127]  M. D’Esposito,et al.  Medial Temporal Lobe Activity Associated with Active Maintenance of Novel Information , 2001, Neuron.

[128]  Seralynne D Vann,et al.  Extensive cytotoxic lesions of the rat retrosplenial cortex reveal consistent deficits on tasks that tax allocentric spatial memory. , 2002, Behavioral neuroscience.

[129]  Katherine M. Mathis Semantic interference from objects both in and out of a scene context. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[130]  R. Malach,et al.  The topography of high-order human object areas , 2002, Trends in Cognitive Sciences.

[131]  P. Perona,et al.  Rapid natural scene categorization in the near absence of attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[132]  C. Harris,et al.  Sentence Context, Word Recognition, and Repetition Blindness , 2002 .

[133]  Russell A. Poldrack,et al.  Putting names to faces: Successful encoding of associative memories activates the anterior hippocampal formation , 2003, NeuroImage.

[134]  A. Dale,et al.  Distinct Patterns of Neural Modulation during the Processing of Conceptual and Syntactic Anomalies , 2003, Journal of Cognitive Neuroscience.

[135]  M. Bar A Cortical Mechanism for Triggering Top-Down Facilitation in Visual Object Recognition , 2003, Journal of Cognitive Neuroscience.

[136]  M. Bar,et al.  Cortical Analysis of Visual Context , 2003, Neuron.

[137]  Yasushi Miyashita,et al.  Forward Processing of Long-Term Associative Memory in Monkey Inferotemporal Cortex , 2003, The Journal of Neuroscience.

[138]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[139]  Valerie A. Carr,et al.  Spatiotemporal Dynamics of Modality-Specific and Supramodal Word Processing , 2003, Neuron.

[140]  Hans-Jochen Heinze,et al.  Human Hippocampal and Parahippocampal Activity during Visual Associative Recognition Memory for Spatial and Nonspatial Stimulus Configurations , 2003, The Journal of Neuroscience.

[141]  Mieke Verfaellie,et al.  Disproportionate deficit in associative recognition relative to item recognition in global amnesia , 2003, Cognitive, affective & behavioral neuroscience.

[142]  H. Spiers,et al.  Prefrontal and medial temporal lobe interactions in long-term memory , 2003, Nature Reviews Neuroscience.

[143]  M. Cherrier,et al.  Agnosia for scenes in topographagnosia , 2003, Neuropsychologia.

[144]  Paul E. Downing,et al.  Viewpoint-Specific Scene Representations in Human Parahippocampal Cortex , 2003, Neuron.

[145]  M. Kutas,et al.  An electrophysiological study of scene effects on object identification. , 2003, Brain research. Cognitive brain research.

[146]  John Lisman,et al.  Recognition by top-down and bottom-up processing in cortex: the control of selective attention. , 2003, Journal of neurophysiology.

[147]  L. Chelazzi,et al.  Associative knowledge controls deployment of visual selective attention , 2003, Nature Neuroscience.

[148]  R. Clark,et al.  The medial temporal lobe. , 2004, Annual review of neuroscience.

[149]  Daniel L Schacter,et al.  Encoding activity in anterior medial temporal lobe supports subsequent associative recognition , 2004, NeuroImage.

[150]  Stefano Soatto,et al.  Deformotion: Deforming Motion, Shape Average and the Joint Registration and Approximation of Structures in Images , 2003, International Journal of Computer Vision.

[151]  Lee Ryan,et al.  An fMRI study of episodic memory: retrieval of object, spatial, and temporal information. , 2004, Behavioral neuroscience.

[152]  Rebecca D Burwell,et al.  Corticohippocampal Contributions to Spatial and Contextual Learning , 2004, The Journal of Neuroscience.

[153]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[154]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[155]  Ethan M. Meyers,et al.  Contextually Evoked Object-Specific Responses in Human Visual Cortex , 2004, Science.

[156]  Antonio Torralba,et al.  Contextual Priming for Object Detection , 2003, International Journal of Computer Vision.

[157]  Emmanuel Dupoux,et al.  Partial Awareness Creates the “Illusion” of Subliminal Semantic Priming , 2004, Psychological science.

[158]  S. Zeki,et al.  Functional brain mapping during free viewing of natural scenes , 2004, Human brain mapping.

[159]  Jodi L. Davenport,et al.  Scene Consistency in Object and Background Perception , 2004, Psychological science.

[160]  Michael S. Ambinder,et al.  Change blindness , 1997, Trends in Cognitive Sciences.

[161]  Russell A. Epstein The cortical basis of visual scene processing , 2005 .

[162]  E. Halgren,et al.  Top-down facilitation of visual recognition. , 2006, Proceedings of the National Academy of Sciences of the United States of America.