The signer as an embodied mirror neuron system: neural mechanisms underlying sign language and action

Introduction “Mirror” neurons are found in area F5 of the monkey brain, and they fire both when a monkey grasps an object and when the monkey observes another individual grasping the object (e.g., Rizzolatti et al ., 1996; see Arbib, Chapter 1, this volume, for further discussion). Mirror neurons have also been found in the rostral part of the monkey inferior parietal lobule (Gallese et al ., 2002). Like mirror neurons, signers must associate the visually perceived manual actions of another signer with self-generated actions of the same form. Sign language comprehension and production requires a direct coupling between action observation and action execution. However, unlike mirror neurons for hand movements recorded in monkey, signing is not tied to object manipulation. Mirror neurons for grasping in monkey fire only when an object is present or understood to be present and do not fire when just the grasping movement is presented (Umilta et al ., 2001). Furthermore, unlike grasping and reaching movements, sign articulations are structured within a phonological system of contrasts. The hand configuration for a sign is determined by a phonological specification stored in the lexicon, not by the properties of an object to be grasped. These facts have interesting implications for the evolution of language and for the neural systems that underlie sign language and action. The fact that sign language exhibits form-based patterning of meaningless elements (i.e., phonology) distinguishes signs from actions, even when the two appear quite similar on the surface; for example, the American Sign Language (ASL) sign TO-HAMMER resembles the act of hammering.

[1]  Penny A. MacDonald,et al.  The role of parietal cortex in awareness of self-generated movements: a transcranial magnetic stimulation study. , 2003, Cerebral cortex.

[2]  W. Krieg Functional Neuroanatomy , 1953, Springer Series in Experimental Entomology.

[3]  P. Ladefoged A course in phonetics , 1975 .

[4]  G. Rizzolatti,et al.  Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study , 2001, The European journal of neuroscience.

[5]  D. Brentari,et al.  Categorical perception in American Sign Language , 2003 .

[6]  J. Risberg,et al.  Regional Cerebral Blood Flow in Sign Language Users , 1993, Brain and Language.

[7]  Diane Brentari,et al.  A Prosodic Model of Sign Language Phonology , 1999 .

[8]  J P Rauschecker,et al.  Hemispheric specialization for English and ASL: left invariance‐right variability , 1998, Neuroreport.

[9]  Kevin P. Hinshaw,et al.  Functional Roles of Broca's Area and SMG: Evidence from Cortical Stimulation Mapping in a Deaf Signer , 1999, NeuroImage.

[10]  K. Amunts,et al.  Broca's region , 2006 .

[11]  M. Tomasello,et al.  Imitative learning of actions on objects by children, chimpanzees, and enculturated chimpanzees. , 1993, Child development.

[12]  G. Hickok,et al.  Role of the left hemisphere in sign language comprehension , 2002, Brain and Language.

[13]  R. Campbell,et al.  Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech , 2003, Journal of Cognitive Neuroscience.

[14]  D Bavelier,et al.  Cerebral organization for language in deaf and hearing subjects: biological constraints and effects of experience. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[15]  K. Emmorey,et al.  Language, Gesture, and Space. , 1996 .

[16]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[17]  W. Prinz Common mechanisms in perception and action. , 2002 .

[18]  P. McGuire,et al.  Silent speechreading in the absence of scanner noise: an event‐related fMRI study , 2000, Neuroreport.

[19]  Karen Emmorey,et al.  Neural systems underlying lexical retrieval for sign language , 2003, Neuropsychologia.

[20]  K M Heilman,et al.  Pantomime comprehension and ideomotor apraxia. , 1985, Journal of neurology, neurosurgery, and psychiatry.

[21]  Y. Sugita,et al.  Auditory-visual speech perception examined by fMRI and PET , 2003, Neuroscience Research.

[22]  Diane E. Anderson,et al.  The puzzle of negation: How children move from communicative to grammatical negation in ASL , 1997, Applied Psycholinguistics.

[23]  K. Emmorey Language, Cognition, and the Brain: Insights From Sign Language Research , 2001 .

[24]  K. Emmorey The Role of Broca’s Area in Sign Language , 2006 .

[25]  P. McGuire,et al.  Neural systems underlying British Sign Language and audio-visual English processing in native users. , 2002, Brain : a journal of neurology.

[26]  V. Mountcastle,et al.  Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space. , 1975, Journal of neurophysiology.

[27]  B. Mazoyer,et al.  A Common Language Network for Comprehension and Production: A Contribution to the Definition of Language Epicenters with PET , 2000, NeuroImage.

[28]  W. Levelt,et al.  The spatial and temporal signatures of word production components , 2004, Cognition.

[29]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[30]  Steven L. Small,et al.  Listening to talking faces: motor cortical activation during speech perception , 2005, NeuroImage.

[31]  L. Goldstein,et al.  Articulatory Phonology: A phonology for public language use , 2003 .

[32]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[33]  Ann Senghas,et al.  Intergenerational influence and ontogenetic development in the emergence of spatial grammar in Nicaraguan Sign Language , 2003 .

[34]  R E Carson,et al.  Altered patterns of cerebral activity during speech and language production in developmental stuttering. An H2(15)O positron emission tomography study. , 1997, Brain : a journal of neurology.

[35]  Sarah Florence Taub,et al.  Language from the Body: Iconicity and Metaphor in American Sign Language , 2001 .

[36]  Norihiro Sadato,et al.  Age-dependent plasticity in the superior temporal sulcus in deaf humans: a functional MRI study , 2004, BMC Neuroscience.

[37]  M. Clegg,et al.  Paying the price of speech? An analysis of mortality statistics for choking on food , 2000 .

[38]  Kuniyoshi L Sakai,et al.  Sign and speech: amodal commonality in left hemisphere dominance for comprehension of sentences. , 2005, Brain : a journal of neurology.

[39]  Michael C. Corballis,et al.  From Hand to Mouth: The Origins of Language , 2002 .

[40]  Ursula Bellugi,et al.  What's right about the neural organization of sign language? A perspective on recent neuroimaging results , 1998, Trends in Cognitive Sciences.

[41]  N. A. Borghese,et al.  A functional-anatomical model for lipreading. , 2003, Journal of neurophysiology.

[42]  Kearsy Cormier,et al.  Modality and structure in signed and spoken languages: Frontmatter , 2002 .

[43]  David Quinto-Pozos,et al.  Contact between Mexican sign language and American sign language in two Texas border areas , 2006 .

[44]  D. Slobin The Crosslinguistic Study of Language Acquisition , 1987 .

[45]  W. Sandler,et al.  On the nature of phonological structure in sign language , 1993, Phonology.

[46]  H. Nusbaum,et al.  The development of speech perception : the transition from speech sounds to spoken words , 1997 .

[47]  Alvin M. Liberman,et al.  Speech: A Special Code , 1996 .

[48]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[49]  Toshio Inui,et al.  Cortical involvement for action imitation of hand/arm postures versus finger configurations: an fMRI study , 2002, Neuroreport.

[50]  K. Emmorey,et al.  Lexical Recognition in Sign Language: Effects of Phonetic Structure and Morphology , 1990, Perceptual and motor skills.

[51]  Anne Cutler Speaking for listening , 1987 .

[52]  Hanna Damasio,et al.  Premotor and Prefrontal Correlates of Category-Related Lexical Retrieval , 1998, NeuroImage.

[53]  M. Sereno,et al.  Neural organization for recognition of grammatical and emotional facial expressions in deaf ASL signers and hearing nonsigners. , 2005, Brain research. Cognitive brain research.

[54]  E. Bullmore,et al.  Activation of auditory cortex during silent lipreading. , 1997, Science.

[55]  C. Fowler An event approach to the study of speech perception from a direct realist perspective , 1986 .

[56]  G. Rizzolatti,et al.  I Know What You Are Doing A Neurophysiological Study , 2001, Neuron.

[57]  S D Fischer,et al.  Looking through phonological shape to lexical meaning: The bottleneck of non-native sign language processing , 1989, Memory & cognition.

[58]  A. Braun,et al.  Activation of Broca’s area during the production of spoken and signed language: a combined cytoarchitectonic mapping and PET analysis , 2003, Neuropsychologia.

[59]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[60]  Willem J. M. Levelt,et al.  The neural correlates of language production , 2000 .

[61]  A. Braun,et al.  Neural systems for sign language production: Mechanisms supporting lexical selection, phonological encoding, and articulation , 2004, Human brain mapping.

[62]  J Rönnberg,et al.  Signed and spoken language perception studied by positron emission tomography , 1997, Neurology.

[63]  R Hari,et al.  Cortical representation of sign language: comparison of deaf signers and hearing non-signers. , 2001, Cerebral cortex.

[64]  Peter D. Eimas,et al.  Perspectives on the study of speech , 1981 .

[65]  A. Damasio,et al.  Neural Correlates of Naming Actions and of Naming Spatial Relations , 2001, NeuroImage.

[66]  W. Sandler Phonological Representation of the Sign: Linearity and Nonlinearity in American Sign Language , 1989 .

[67]  Aaron J. Newman,et al.  A critical period for right hemisphere recruitment in American Sign Language processing , 2002, Nature Neuroscience.

[68]  P. Sinha,et al.  Functional neuroanatomy of biological motion perception in humans , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[69]  A. Damasio,et al.  A neural basis for lexical retrieval , 1996, Nature.

[70]  Scott T. Grafton,et al.  Premotor Cortex Activation during Observation and Naming of Familiar Tools , 1997, NeuroImage.

[71]  Naoki Miura,et al.  The human parietal cortex is involved in spatial processing of tongue movement—an fMRI study , 2004, NeuroImage.

[72]  Anne Cutler,et al.  Twenty-first century psycholinguistics : four cornerstones , 2005 .

[73]  K. Isselbacher,et al.  Heterozygous germ line hCHK2 mutations in Li-Fraumeni syndrome. , 1999, Science.

[74]  C. Frith,et al.  Functional neuroanatomy of verbal self-monitoring , 1996, NeuroImage.

[75]  J. Mazziotta,et al.  Grasping the Intentions of Others with One's Own Mirror Neuron System , 2005, PLoS biology.

[76]  G. Rizzolatti,et al.  Motor and cognitive functions of the ventral premotor cortex , 2002, Current Opinion in Neurobiology.

[77]  Laura A. Petitto,et al.  On the autonomy of language and gesture: Evidence from the acquisition of personal pronouns in American sign language , 1987, Cognition.

[78]  R J Wise,et al.  Separate neural subsystems within 'Wernicke's area'. , 2001, Brain : a journal of neurology.

[79]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[80]  Alan C. Evans,et al.  Speech-like cerebral activity in profoundly deaf people processing signed languages: implications for the neural basis of human language. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[81]  K. Heilman,et al.  Two forms of ideomotor apraxia , 1982, Neurology.

[82]  D. Wolpert,et al.  Maintaining internal representations: the role of the human superior parietal lobe , 1998, Nature Neuroscience.

[83]  D. Na,et al.  Functional magnetic resonance imaging during pantomiming tool-use gestures , 2001, Experimental Brain Research.

[84]  Philip K. McGuire,et al.  Dissociating linguistic and nonlinguistic gestural communication in the brain , 2004, NeuroImage.

[85]  Elissa L. Newport,et al.  The acquisition of American Sign Language. , 1985 .

[86]  Patricia Siple,et al.  Understanding language through sign language research , 1978 .

[87]  Karen Emmorey,et al.  Motor-iconicity of sign language does not alter the neural systems underlying tool and action naming , 2004, Brain and Language.

[88]  P. McGuire,et al.  Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). , 2001, Brain research. Cognitive brain research.

[89]  M. Gazzaniga,et al.  The new cognitive neurosciences , 2000 .

[90]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[91]  M. Arbib From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics , 2005, Behavioral and Brain Sciences.

[92]  K. Hashikawa,et al.  Sign language ‘heard’ in the auditory cortex , 1999, Nature.

[93]  U. Bellugi,et al.  Dissociation between linguistic and nonlinguistic gestural systems: A case for compositionality , 1992, Brain and Language.

[94]  J. Decety,et al.  From the perception of action to the understanding of intention , 2001, Nature reviews. Neuroscience.

[95]  S Iwaki,et al.  Neural substrates involved in imitating finger configurations: an fMRI study , 2001, Neuroreport.

[96]  Bencie Woll,et al.  Aphasia in a user of British Sign Language: Dissociation between sign and gesture , 2004, Cognitive neuropsychology.

[97]  J. Moll,et al.  Functional MRI correlates of real and imagined tool-use pantomimes , 2000, Neurology.

[98]  R. Blake,et al.  Brain Areas Involved in Perception of Biological Motion , 2000, Journal of Cognitive Neuroscience.

[99]  A. Braun,et al.  The neural organization of discourse: an H2 15O-PET study of narrative production in English and American sign language. , 2001, Brain : a journal of neurology.

[100]  Antje S. Meyer,et al.  Phonetics and Phonology in Language Comprehension and Production: Differences and Similarities , 2003, Phonetica.

[101]  R. Passingham,et al.  Action observation and acquired motor skills: an FMRI study with expert dancers. , 2005, Cerebral cortex.

[102]  Onno Crasborn,et al.  Phonetic implementation of phonological categories in Sign Language of the Netherlands , 2001 .

[103]  Terry Janzen,et al.  Modality and structure in signed and spoken languages: Gesture as the substrate in the process of ASL grammaticization , 2002 .

[104]  Ione Fine,et al.  Visual stimuli activate auditory cortex in the deaf , 2001, Nature Neuroscience.