Thinking about seeing: Perceptual sources of knowledge are encoded in the theory of mind brain regions of sighted and blind adults

Blind people's inferences about how other people see provide a window into fundamental questions about the human capacity to think about one another's thoughts. By working with blind individuals, we can ask both what kinds of representations people form about others' minds, and how much these representations depend on the observer having had similar mental states themselves. Thinking about others' mental states depends on a specific group of brain regions, including the right temporo-parietal junction (RTPJ). We investigated the representations of others' mental states in these brain regions, using multivoxel pattern analyses (MVPA). We found that, first, in the RTPJ of sighted adults, the pattern of neural response distinguished the source of the mental state (did the protagonist see or hear something?) but not the valence (did the protagonist feel good or bad?). Second, these neural representations were preserved in congenitally blind adults. These results suggest that the temporo-parietal junction contains explicit, abstract representations of features of others' mental states, including the perceptual source. The persistence of these representations in congenitally blind adults, who have no first-person experience with sight, provides evidence that these representations emerge even in the absence of relevant first-person perceptual experiences.

[1]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[2]  Lindsey J. Powell,et al.  It's the Thought That Counts , 2006, Psychological science.

[3]  Brenda Schick,et al.  Language and theory of mind: a study of deaf children. , 2007, Child development.

[4]  M. Peelen,et al.  Supramodal Representations of Perceived Emotions in the Human Brain , 2010, The Journal of Neuroscience.

[5]  R. Saxe,et al.  Decoding moral judgments from neural representations of intentions , 2013, Proceedings of the National Academy of Sciences.

[6]  R Saxe,et al.  People thinking about thinking people The role of the temporo-parietal junction in “theory of mind” , 2003, NeuroImage.

[7]  Chun-I Yeh,et al.  Temporal precision in the neural code and the timescales of natural vision , 2007, Nature.

[8]  C. Frith,et al.  Action Observation: Inferring Intentions without Mirror Neurons , 2008, Current Biology.

[9]  Andrew N. Meltzoff,et al.  10 Eyes Wide Shut: The Importance of Eyes in Infant Gaze Following and Understanding Other Minds , 2006 .

[10]  Martina Faller Semantics and pragmatics of evidentials in Cuzco Quechua , 2002 .

[11]  A. Gopnik,et al.  Words, thoughts, and theories , 1997 .

[12]  M. Just,et al.  Differentiable cortical networks for inferences concerning people's intentions versus physical causality , 2011, Human brain mapping.

[13]  S Millar,et al.  Spatial representation by blind and sighted children. , 1976, Journal of experimental child psychology.

[14]  R. Goebel,et al.  Mirror-Symmetric Tonotopic Maps in Human Primary Auditory Cortex , 2003, Neuron.

[15]  E. Borgida,et al.  Juror Decision Making and the Evaluation of Hearsay Evidence , 1992 .

[16]  N. Kanwisher,et al.  Lexical and syntactic representations in the brain: An fMRI investigation with multi-voxel pattern analyses , 2011, Neuropsychologia.

[17]  L. Shah,et al.  Functional magnetic resonance imaging. , 2010, Seminars in roentgenology.

[18]  J. de Villiers,et al.  Evidentials in Tibetan: acquisition, semantics, and cognitive development. , 2009, New directions for child and adolescent development.

[19]  R. Passingham,et al.  Action observation and acquired motor skills: an FMRI study with expert dancers. , 2005, Cerebral cortex.

[20]  Josef Perner,et al.  Temporo-parietal Junction Activity in Theory-of-Mind Tasks: Falseness, Beliefs, or Attention , 2009, Journal of Cognitive Neuroscience.

[21]  Paul L. Harris,et al.  From Simulation to Folk Psychology: The Case for Development , 1992 .

[22]  de Villiers,et al.  The Role of Language in Theory-of-Mind Development: What Deaf Children Tell Us. , 2005 .

[23]  Bradford Z. Mahon,et al.  Judging semantic similarity: an event-related fMRI study with auditory word stimuli , 2010, Neuroscience.

[24]  P. Harris,et al.  Trust in testimony: how children learn about science and religion. , 2006, Child development.

[25]  Edmund T. Rolls,et al.  The neuronal encoding of information in the brain , 2011, Progress in Neurobiology.

[26]  安平鎬,et al.  Evidentiality , 2018, A Grammar of Nganasan.

[27]  Anna Papafragou,et al.  Evidential Morphology and Theory of Mind , 2002 .

[28]  M. P. Moeller,et al.  Relations between maternal input and theory of mind understanding in deaf children. , 2006, Child development.

[29]  G. Boynton,et al.  Feature-Based Attentional Modulations in the Absence of Direct Visual Stimulation , 2007, Neuron.

[30]  Ann Senghas,et al.  Language Promotes False-Belief Understanding , 2009, Psychological science.

[31]  Helen Chilton,et al.  Theory of mind development , 2010 .

[32]  Other Minds,et al.  Eyes Wide Shut: The Importance of Eyes in Infant Gaze-Following and Understanding , 2007 .

[33]  Rebecca Saxe,et al.  fMRI item analysis in a theory of mind task , 2011, NeuroImage.

[34]  C. Keysers,et al.  The Observation and Execution of Actions Share Motor and Somatosensory Voxels in all Tested Subjects: Single-Subject Analyses of Unsmoothed fMRI Data , 2008, Cerebral cortex.

[35]  Ian A. Apperly,et al.  Beyond Simulation–Theory and Theory–Theory: Why social cognitive neuroscience should use its own concepts to study “theory of mind” , 2008, Cognition.

[36]  Anastasia Smirnova,et al.  Evidentiality in Bulgarian: Temporality, Epistemic Modality, and Information Source , 2013, J. Semant..

[37]  G. S. Marmor Age at onset of blindness and the development of the semantics of color names. , 1978, Journal of experimental child psychology.

[38]  Amishi P. Jha,et al.  Viewing facial expressions of pain engages cortical areas involved in the direct experience of pain , 2005, NeuroImage.

[39]  Alessandro Lenci,et al.  BLIND: a set of semantic feature norms from the congenitally blind , 2013, Behavior research methods.

[40]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[41]  R. Gordon Folk Psychology as Simulation , 1986 .

[42]  A. Goldman,et al.  Mirror neurons and the simulation theory of mind-reading , 1998, Trends in Cognitive Sciences.

[43]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[44]  A. Meltzoff 'Like me': a foundation for social cognition. , 2007, Developmental science.

[45]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[46]  Matthew D. Lieberman,et al.  Identifying the What, Why, and How of an Observed Action: An fMRI Study of Mentalizing and Mechanizing during Action Observation , 2011, Journal of Cognitive Neuroscience.

[47]  J. Braithwaite,et al.  Seeing it their way: evidence for rapid and involuntary computation of what other people see. , 2010, Journal of experimental psychology. Human perception and performance.

[48]  P. Harris The Cognitive Basis of Science: What do children learn from testimony? , 2002 .

[49]  J. O'Doherty,et al.  Empathy for Pain Involves the Affective but not Sensory Components of Pain , 2004, Science.

[50]  Stanka A. Fitneva,et al.  Epistemic marking and reliability judgments : Evidence from Bulgarian , 2001 .

[51]  David D. Cox,et al.  Untangling invariant object recognition , 2007, Trends in Cognitive Sciences.

[52]  Yul-Wan Sung,et al.  Functional magnetic resonance imaging , 2004, Scholarpedia.

[53]  R. Shepard,et al.  Representation of Colors in the Blind, Color-Blind, and Normally Sighted , 1992 .

[54]  Edward T. Bullmore,et al.  Specialization of right temporo-parietal junction for mentalizing and its relation to social impairments in autism , 2011, NeuroImage.

[55]  R. Hobson,et al.  Congenital visual impairment and 'theory of mind' , 1998 .

[56]  Amy Coplan Simulating Minds: The Philosophy, Psychology, and Neuroscience of Mindreading by goldman, alvin , 2008 .

[57]  V. Gallese,et al.  What is so special about embodied simulation? , 2011, Trends in Cognitive Sciences.

[58]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[59]  S. Stich,et al.  The cognitive basis of science , 2002 .

[60]  A. Gopnik,et al.  Young Children's Ability to Identify the Sources of Their Beliefs. , 1991 .

[61]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[62]  H. Wellman The Child's Theory of Mind , 1990 .

[63]  J. Rosel,et al.  Verbalism in the Narrative Language of Children who are Blind and Sighted , 2005 .

[64]  Margaret Bull Kovera,et al.  Jurors' Perceptions of Eyewitness and Hearsay Evidence , 1992 .

[65]  Y. Paulignan,et al.  An Interference Effect of Observed Biological Movement on Action , 2003, Current Biology.

[66]  Alison J. Wiggett,et al.  Patterns of fMRI Activity Dissociate Overlapping Functional Brain Areas that Respond to Biological Motion , 2006, Neuron.

[67]  Michael Siegal,et al.  Representing Inner Worlds: Theory of Mind in Autistic, Deaf, and Normal Hearing Children , 1999 .

[68]  R. Saxe,et al.  Insights into the origins of knowledge from the cognitive neuroscience of blindness , 2012, Cognitive neuropsychology.

[69]  H. Wellman,et al.  From fancy to reason: scaling deaf and hearing children's understanding of theory of mind and pretence. , 2009, The British journal of developmental psychology.

[70]  Robbe L. T. Goris,et al.  Frontiers in Computational Neuroscience Computational Neuroscience Neural Representations That Support Invariant Object Recognition , 2022 .

[71]  A. Bigelow Blind Children's Ability to Predict What Another Sees , 1992 .

[72]  P. Harris,et al.  Trust in Testimony: Children's Use of True and False Statements , 2004, Psychological science.

[73]  S. Stich,et al.  The Varieties of Off-Line Simulation , 1996 .

[74]  C. L. Moore,et al.  The Development of Social Understanding in Children with Visual Impairments , 1995 .

[75]  S. Dehaene,et al.  Cultural Recycling of Cortical Maps , 2007, Neuron.

[76]  H. L. Owrid Language acquisition in the blind child: Normal and deficient , 1984 .

[77]  S-J Blakemore,et al.  Somatosensory activations during the observation of touch and a case of vision-touch synaesthesia. , 2005, Brain : a journal of neurology.

[78]  Erika Nurmsoo,et al.  Children's working understanding of knowledge sources: Confidence in knowledge gained from testimony , 2008 .

[79]  R. Passingham,et al.  Seeing or Doing? Influence of Visual and Motor Familiarity in Action Observation , 2006, Current Biology.

[80]  A. Damasio,et al.  Neural correlates of admiration and compassion , 2009, Proceedings of the National Academy of Sciences.

[81]  K. W. Falkman,et al.  Language access and theory of mind reasoning: evidence from deaf children in bilingual and oralist environments. , 2007, Developmental psychology.

[82]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[83]  Alvaro Pascual-Leone,et al.  Growing up blind does not change the neural bases of Theory of Mind , 2009, Proceedings of the National Academy of Sciences.

[84]  Donald T. Stuss,et al.  Common and Unique Neural Correlates of Autobiographical Memory and Theory of Mind , 2010, Journal of Cognitive Neuroscience.

[85]  Nikolaus Kriegeskorte,et al.  Analyzing for information, not activation, to exploit high-resolution fMRI , 2007, NeuroImage.

[86]  S. Stich,et al.  Folk Psychology: Simulation or Tacit Theory? , 1992 .

[87]  E. Robinson,et al.  Children's suggestibility in relation to their understanding about sources of knowledge. , 2003, Child development.

[88]  M. Brambring,et al.  Validity of False Belief Tasks in Blind Children , 2010, Journal of autism and developmental disorders.

[89]  R. Saxe Against simulation: the argument from error , 2005, Trends in Cognitive Sciences.

[90]  Rebecca Saxe,et al.  To peek and to peer: "visual" verb meanings are largely unaffected by congenital blindnsess , 2012, CogSci.

[91]  J. Perner Understanding the Representational Mind , 1993 .

[92]  C. Peterson,et al.  Factors influencing the development of a theory of mind in blind children , 2000 .

[93]  S. Thompson-Schill,et al.  Semantic adaptation and competition during word comprehension. , 2008, Cerebral cortex.

[94]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[95]  P. Harris Checking our sources: the origins of trust in testimony , 2002 .

[96]  C. Keysers,et al.  Integrating simulation and theory of mind: from self to social cognition , 2007, Trends in Cognitive Sciences.

[97]  P. Harris,et al.  Theory of mind development in deaf children: a nonverbal test of false-belief understanding. , 2001, Journal of deaf studies and deaf education.

[98]  Andrew T. Smith,et al.  The confounding effect of response amplitude on MVPA performance measures , 2011, NeuroImage.

[99]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[100]  A. Gopnik,et al.  Why the Child's Theory of Mind Really Is a Theory , 1992 .

[101]  B. Landau Language and experience , 1985 .

[102]  Michael Siegal,et al.  Signposts to development: theory of mind in deaf children. , 2002, Child development.

[103]  A. Bigelow Hiding in blind and sighted children , 1991, Development and Psychopathology.