Communicative Signals Promote Object Recognition Memory and Modulate the Right Posterior STS

Detection of communicative signals is thought to facilitate knowledge acquisition early in life, but less is known about the role these signals play in adult learning or about the brain systems supporting sensitivity to communicative intent. The current study examined how ostensive gaze cues and communicative actions affect adult recognition memory and modulate neural activity as measured by fMRI. For both the behavioral and fMRI experiments, participants viewed a series of videos of an actress acting on one of two objects in front of her. Communicative context in the videos was manipulated in a 2 × 2 design in which the actress either had direct gaze (Gaze) or wore a visor (NoGaze) and either pointed at (Point) or reached for (Reach) one of the objects (target) in front of her. Participants then completed a recognition memory task with old (target and nontarget) objects and novel objects. Recognition memory for target objects in the Gaze conditions was greater than NoGaze, but no effects of gesture type were seen. Similarly, the fMRI video-viewing task revealed a significant effect of Gaze within right posterior STS (pSTS), but no significant effects of Gesture. Furthermore, pSTS sensitivity to Gaze conditions was related to greater memory for objects viewed in Gaze, as compared with NoGaze, conditions. Taken together, these results demonstrate that the ostensive, communicative signal of direct gaze preceding an object-directed action enhances recognition memory for attended items and modulates the pSTS response to object-directed actions. Thus, establishment of a communicative context through ostensive signals remains an important component of learning and memory into adulthood, and the pSTS may play a role in facilitating this type of social learning.

[1]  Gian Luca Romani,et al.  Brain activity modulation during the production of imperative and declarative pointing , 2015, NeuroImage.

[2]  Linda B. Smith,et al.  Joint Attention without Gaze Following: Human Infants and Their Parents Coordinate Visual Attention to Objects through Eye-Hand Coordination , 2013, PloS one.

[3]  P. Mundy,et al.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Attention, Joint Attention, and Social Cognition , 2022 .

[4]  Stefanie Hoehl,et al.  Neural mechanisms of joint attention in infancy , 2006, The European journal of neuroscience.

[5]  M. Brass,et al.  Investigating Action Understanding: Inferential Processes versus Action Simulation , 2007, Current Biology.

[6]  M. Gentilucci,et al.  The eye contact effect in request and emblematic hand gestures , 2014, The European journal of neuroscience.

[7]  Peter Thier,et al.  Dissociable Roles of the Superior Temporal Sulcus and the Intraparietal Sulcus in Joint Attention: A Functional Magnetic Resonance Imaging Study , 2008, Journal of Cognitive Neuroscience.

[8]  D. Sperber,et al.  Relevance: Communication and Cognition , 1989 .

[9]  Nadim Joni Shah,et al.  Minds Made for Sharing: Initiating Joint Attention Recruits Reward-related Neurocircuitry , 2010, Journal of Cognitive Neuroscience.

[10]  M. Tomasello,et al.  A new look at infant pointing. , 2007, Child development.

[11]  Frank Van Overwalle,et al.  Understanding others' actions and goals by mirror and mentalizing systems: A meta-analysis , 2009, NeuroImage.

[12]  Robert P. Spunt,et al.  Dissociable Neural Systems Support Retrieval of How and Why Action Knowledge , 2010, Psychological science.

[13]  Matthew D. Lieberman,et al.  Identifying the What, Why, and How of an Observed Action: An fMRI Study of Mentalizing and Mechanizing during Action Observation , 2011, Journal of Cognitive Neuroscience.

[14]  R. Saxe,et al.  Look at this: the neural correlates of initiating and responding to bids for joint attention , 2012, Front. Hum. Neurosci..

[15]  C. Frith,et al.  Meeting of minds: the medial frontal cortex and social cognition , 2006, Nature Reviews Neuroscience.

[16]  Nadim Joni Shah,et al.  Duration matters: Dissociating neural correlates of detection and evaluation of social gaze , 2009, NeuroImage.

[17]  Michael Morales,et al.  Responding to Joint Attention Across the 6- Through 24-Month Age Period and Early Language Acquisition , 2000 .

[18]  Rainer Goebel,et al.  Playing Charades in the fMRI: Are Mirror and/or Mentalizing Areas Involved in Gestural Communication? , 2009, PloS one.

[19]  K. Pelphrey,et al.  Brain Mechanisms for Social Perception , 2008, Annals of the New York Academy of Sciences.

[20]  M. Tomasello,et al.  One-year-olds comprehend the communicative intentions behind gestures in a hiding game. , 2005, Developmental science.

[21]  Kimberly J. Montgomery,et al.  Communicative hand gestures and object-directed hand movements activated the mirror neuron system. , 2007, Social cognitive and affective neuroscience.

[22]  A. Berthoz,et al.  Behavioral and Neural Correlates of Communication via Pointing , 2011, PloS one.

[23]  G. Csibra,et al.  Nonverbal communicative signals modulate attention to object properties. , 2014, Journal of experimental psychology. Human perception and performance.

[24]  S. Levinson,et al.  Brain Mechanisms Underlying Human Communication , 2009, Front. Hum. Neurosci..

[25]  H. Walter,et al.  Do you mean me? Communicative intentions recruit the mirror and the mentalizing system. , 2014, Social cognitive and affective neuroscience.

[26]  Kristian Tylén,et al.  Interaction vs. observation: distinctive modes of social cognition in human brain and behavior? A combined fMRI and eye-tracking study , 2012, Front. Hum. Neurosci..

[27]  Sterling C. Johnson,et al.  A generalized form of context-dependent psychophysiological interactions (gPPI): A comparison to standard approaches , 2012, NeuroImage.

[28]  T. Allison,et al.  Brain activation evoked by perception of gaze shifts: the influence of context , 2003, Neuropsychologia.

[29]  G. Rizzolatti,et al.  ß Federation of European Neuroscience Societies Mirror , 2003 .

[30]  P. Mundy,et al.  Affective sharing in the context of joint attention interactions of normal, autistic, and mentally retarded children , 1990, Journal of autism and developmental disorders.

[31]  D. Sperber,et al.  Relevance: Communication and Cognition , 1997 .

[32]  Kwanguk Kim,et al.  Joint Attention, Social-Cognition, and Recognition Memory in Adults , 2012, Front. Hum. Neurosci..

[33]  K. Pelphrey,et al.  An integrative neural model of social perception, action observation, and theory of mind , 2015, Neuroscience & Biobehavioral Reviews.

[34]  Michael Tomasello,et al.  Infants use shared experience to interpret pointing gestures. , 2009, Developmental science.

[35]  C. Frith,et al.  “Hey John”: Signals Conveying Communicative Intention toward the Self Activate Brain Regions Associated with “Mentalizing,” Regardless of Modality , 2003, The Journal of Neuroscience.

[36]  Rebecca Saxe,et al.  Live face-to-face interaction during fMRI: A new tool for social cognitive neuroscience , 2010, NeuroImage.

[37]  F. Volkmar,et al.  The enactive mind, or from actions to cognition: lessons from autism. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[38]  Jennifer M. D. Yoon,et al.  Communication-induced memory biases in preverbal infants , 2008, Proceedings of the National Academy of Sciences.

[39]  S. Corkin,et al.  Two routes to emotional memory: distinct neural processes for valence and arousal. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[40]  C. Frith,et al.  Rapid Response Subject Collections , 2022 .

[41]  Stefan Heim,et al.  Differential role of the Mentalizing and the Mirror Neuron system in the imitation of communicative gestures , 2013, NeuroImage.

[42]  A. Mastergeorge,et al.  A parallel and distributed‐processing model of joint attention, social cognition and autism , 2009, Autism research : official journal of the International Society for Autism Research.

[43]  Felix Bermpohl,et al.  Imagining triadic interactions simultaneously activates mirror and mentalizing systems , 2014, NeuroImage.

[44]  A. Woodward,et al.  Why Do Child-Directed Interactions Support Imitative Learning in Young Children? , 2014, PloS one.

[45]  E. Redcay.  The superior temporal sulcus performs a common function for social and speech perception: Implications for the emergence of autism , 2008, Neuroscience & Biobehavioral Reviews.

[46]  Nathalie Boddaert,et al.  Autism, the superior temporal sulcus and social perception , 2006, Trends in Neurosciences.

[47]  K. Pelphrey,et al.  Research review: Constraining heterogeneity: the social brain and its development in autism spectrum disorder. , 2011, Journal of child psychology and psychiatry, and allied disciplines.

[48]  Stefanie Hoehl,et al.  What are you looking at? Infants' neural processing of an adult's object-directed eye gaze. , 2008, Developmental science.

[49]  Mark H Johnson,et al.  Eye gaze cueing facilitates neural processing of objects in 4-month-old infants , 2004, Neuroreport.

[50]  P. Kuhl,et al.  Foreign-language experience in infancy: Effects of short-term exposure and social interaction on phonetic learning , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[51]  U. Lindenberger,et al.  Effects of joint attention on long-term memory in 9-month-old infants: an event-related potentials study. , 2011, Developmental science.

[52]  K. Pierce,et al.  Preference for geometric patterns early in life as a risk factor for autism. , 2011, Archives of general psychiatry.

[53]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[54]  G. Fink,et al.  Being with virtual others: Neural correlates of social interaction , 2006, Neuropsychologia.

[55]  G. McCarthy,et al.  When Strangers Pass , 2004, Psychological science.

[56]  J. Mazziotta,et al.  Grasping the Intentions of Others with One's Own Mirror Neuron System , 2005, PLoS biology.

[57]  R. Saxe,et al.  Do You See What I See? The Neural Bases of Joint Attention , 2013 .

[58]  T. Striano,et al.  Adult gaze influences infant attention and object processing: implications for cognitive neuroscience , 2005, The European journal of neuroscience.

[59]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[60]  G. Dawson.  Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder , 2008, Development and Psychopathology.

[61]  Brian Knutson,et al.  Reward-Motivated Learning: Mesolimbic Activation Precedes Memory Formation , 2006, Neuron.

[62]  M. Tomasello,et al.  Understanding and sharing intentions: The origins of cultural cognition , 2005, Behavioral and Brain Sciences.

[63]  Gergely Csibra,et al.  Recognizing Communicative Intentions in Infancy , 2010 .

[64]  G. Csibra,et al.  Natural pedagogy , 2009, Trends in Cognitive Sciences.