Motion-sensitive cortex and motion semantics in American Sign Language

Previous research indicates that motion-sensitive brain regions are engaged when comprehending motion semantics expressed by words or sentences. Using fMRI, we investigated whether such neural modulation can occur when the linguistic signal itself is visually dynamic and motion semantics is expressed by movements of the hands. Deaf and hearing users of American Sign Language (ASL) were presented with signed sentences that conveyed motion semantics ("The deer walked along the hillside.") or were static, conveying little or no motion ("The deer slept along the hillside."); sentences were matched for the amount of visual motion. Motion-sensitive visual areas (MT+) were localized individually in each participant. As a control, the Fusiform Face Area (FFA) was also localized for the deaf participants. The whole-brain analysis revealed static (locative) sentences engaged regions in left parietal cortex more than motion sentences, replicating previous results implicating these regions in comprehending spatial language for sign languages. Greater activation was observed in the functionally defined MT+ ROI for motion than static sentences for both deaf and hearing signers. No modulation of neural activity by sentence type was observed in the FFA. Deafness did not affect modulation of MT+ by motion semantics, but hearing signers exhibited stronger neural activity in MT+ for both sentence types, perhaps due to differences in exposure and/or use of ASL. We conclude that top down modulation of motion-sensitive cortex by linguistic semantics is not disrupted by the visual motion that is present in sign language sentences.

[1]  T. Talavage,et al.  Neuroanatomical distribution of five semantic components of verbs: Evidence from fMRI , 2008, Brain and Language.

[2]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[3]  I. Johnsrude,et al.  The problem of functional localization in the human brain , 2002, Nature Reviews Neuroscience.

[4]  K. Emmorey,et al.  Bilingual processing of ASL-English code-blends: The consequences of accessing two lexical representations simultaneously. , 2012, Journal of memory and language.

[5]  Karen R. Dobkins,et al.  Visual Field Asymmetries for Motion Processing in Deaf and Hearing Signers , 2002, Brain and Cognition.

[6]  Terri Gullickson.  Human Brain Anatomy in Computerized Images. , 1995 .

[7]  C. Hutton,et al.  Visual Attention to the Periphery Is Enhanced in Congenitally Deaf Individuals , 2000, The Journal of Neuroscience.

[8]  J. Zacks.  Neuroimaging Studies of Mental Rotation: A Meta-analysis and Review , 2008, Journal of Cognitive Neuroscience.

[9]  F. Pulvermüller,et al.  Walking or Talking?: Behavioral and Neurophysiological Correlates of Action Verb Processing , 2001, Brain and Language.

[10]  Ione Fine,et al.  Comparing the Effects of Auditory Deprivation and Sign Language within the Auditory and Visual Cortex , 2005, Journal of Cognitive Neuroscience.

[11]  H. Neville,et al.  Attention to central and peripheral visual space in a movement detection task. III. Separate effects of auditory deprivation and acquisition of a visual language , 1987, Brain Research.

[12]  Eraldo Paulesu,et al.  The role of age of acquisition and language usage in early, high‐proficient bilinguals: An fMRI study during verbal fluency , 2003, Human brain mapping.

[13]  Karen Emmorey,et al.  Modulation of BOLD Response in Motion-sensitive Lateral Temporal Cortex by Real and Fictive Motion Sentences , 2010, Journal of Cognitive Neuroscience.

[14]  A. Damasio.  Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition , 1989, Cognition.

[15]  Bahador Bahrami,et al.  Visual motion interferes with lexical decision on motion words , 2008, Current Biology.

[16]  B. Bahrami,et al.  Motion Detection and Motion Verbs , 2007, Psychological science.

[17]  Karen Emmorey,et al.  Bimodal bilingualism. , 2008, Bilingualism.

[18]  P. Indefrey.  A Meta‐analysis of Hemodynamic Studies on First and Second Language Processing: Which Suggested Differences Can We Trust and What Do They Mean? , 2006 .

[19]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[20]  Arturo E. Hernandez,et al.  Balancing bilinguals: lexical-semantic production and cognitive processing in children learning Spanish and English. , 1999, Journal of speech, language, and hearing research : JSLHR.

[21]  Angela D. Friederici,et al.  Top-Down and Bottom-Up Contributions to Understanding Sentences Describing Objects in Motion , 2010, Front. Psychology.

[22]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[23]  N. Kanwisher,et al.  The fusiform face area subserves face perception, not generic within-category identification , 2004, Nature Neuroscience.

[24]  K. Grammer,et al.  Dynamic systems and inferential information processing in human communication. , 2002, Neuro endocrinology letters.

[25]  G. Lakoff,et al.  The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge , 2005, Cognitive neuropsychology.

[26]  R. Battison,et al.  Lexical Borrowing in American Sign Language , 1978 .

[27]  H. Damasio.  Human Brain Anatomy in Computerized Images , 1995 .

[28]  W. Stokoe,et al.  Sign language structure: an outline of the visual communication systems of the American deaf. 1960. , 1961, Journal of deaf studies and deaf education.

[29]  Michael P. Kaschak,et al.  Putting words in perspective , 2004, Memory & cognition.

[30]  Bencie Woll,et al.  Phonological processing in deaf signers and the impact of age of first language acquisition , 2008, NeuroImage.

[31]  Diane Brentari,et al.  A Prosodic Model of Sign Language Phonology , 1999 .

[32]  Friedemann Pulvermüller,et al.  Understanding in an instant: Neurophysiological evidence for mechanistic language circuits in the brain , 2009, Brain and Language.

[33]  Michael K. Tanenhaus,et al.  Neural correlates of partial lexical activation , 2008, Proceedings of the National Academy of Sciences.

[34]  Rolf A. Zwaan,et al.  Seeing, acting, understanding: motor resonance in language comprehension. , 2006, Journal of experimental psychology. General.

[35]  Craig J. Brozinsky,et al.  Impact of Early Deafness and Early Exposure to Sign Language on the Cerebral Organization for Motion Processing , 2001, The Journal of Neuroscience.

[36]  A. Schleicher,et al.  Cytoarchitectonic analysis of the human extrastriate cortex in the region of V5/MT+: a probabilistic, stereotaxic map of area hOc5. , 2006, Cerebral cortex.

[37]  Michael P. Kaschak,et al.  Perception of motion affects language processing , 2005, Cognition.

[38]  Richard S. J. Frackowiak,et al.  Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. , 1993, Cerebral cortex.

[39]  Martin Fieder,et al.  The Communication Paradox and Possible Solutions , 1996 .

[40]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[41]  G. Rizzolatti,et al.  Congruent Embodied Representations for Visually Presented Actions and Linguistic Phrases Describing Actions , 2006, Current Biology.

[42]  R. Andersen,et al.  Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[43]  J. Kable,et al.  Neural Substrates of Action Event Knowledge , 2002, Journal of Cognitive Neuroscience.

[44]  Philip K. McGuire,et al.  Neural Correlates of British Sign Language Comprehension: Spatial Processing Demands of Topographic Language , 2002, Journal of Cognitive Neuroscience.

[45]  James C Craig,et al.  Visual Motion Interferes with Tactile Motion Perception , 2006, Perception.