Representing Motion in Language Comprehension: Lessons From Neuroimaging

A central issue in understanding how language links the mental and the real world is the nature of the mental representations entertained during language processing. Are these mental representations closely linked to the perceptual experiences from which they were formed or are they somewhat removed from them? This review addresses this question by examining studies that have investigated motion verbs and sentences using functional magnetic resonance imaging. These studies tested whether language processing elicits modality-specific brain regions responsive to motion perception. Although the results of these studies are not definite due to the different tasks and analysis techniques utilized, they so far suggest that modality-specific brain regions processing visual motion are not automatically or habitually engaged in language processing. The occasional engagement of visual areas in language processing appears to result from tasks requiring integration of visual and linguistic information or attention to motion-specific features such as direction. The evidence reviewed therefore suggests that although perceptual representations may be flexibly engaged as a function of tasks and contexts, language comprehension in the absence of visual contexts habitually engages experience-based representations of motion events that are one-step removed from visual experiences, even in situations in which imagery is encouraged. Language is often used to describe objects and events in the world. In this function, words and sentences elicit mental representations of objects and events that have been

[1]  Friedemann Pulvermüller,et al.  Brain mechanisms linking language and action , 2005, Nature Reviews Neuroscience.

[2]  Silvia P. Gennari,et al.  Semantic indeterminacy in object relative clauses. , 2008, Journal of memory and language.

[3]  Aina Puce,et al.  Configural Processing of Biological Motion in Human Superior Temporal Sulcus , 2005, The Journal of Neuroscience.

[4]  A. Roepstorff,et al.  Motion verb sentences activate left posterior middle temporal cortex despite static context , 2005, Neuroreport.

[5]  J. Haxby,et al.  Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects , 1999, Nature Neuroscience.

[6]  Sharon L. Thompson-Schill,et al.  Conceptual Representations of Action in the Lateral Temporal Cortex , 2005, Journal of Cognitive Neuroscience.

[7]  H. Damasio,et al.  Effects of noun–verb homonymy on the neural correlates of naming concrete entities and actions , 2005, Brain and Language.

[8]  A. Treisman,et al.  Voluntary Attention Modulates fMRI Activity in Human MT–MST , 1997, Neuron.

[9]  Christine D. Wilson,et al.  Grounding conceptual knowledge in modality-specific systems , 2003, Trends in Cognitive Sciences.

[10]  Michael K. Tanenhaus,et al.  Neural correlates of partial lexical activation , 2008, Proceedings of the National Academy of Sciences.

[11]  G. Bower,et al.  Mental models in narrative comprehension. , 1990, Science.

[12]  Aina Puce,et al.  Electrophysiology and brain imaging of biological motion. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  Friedemann Pulvermüller,et al.  Brain Signatures of Meaning Access in Action Word Recognition , 2005, Journal of Cognitive Neuroscience.

[14]  Rieko Osu,et al.  The Neural Substrates of Biological Motion Perception: an fMRI Study , 2022 .

[15]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[16]  R. Adolphs,et al.  NEURAL CORRELATES OF CONCEPTUAL KNOWLEDGE FOR ACTIONS , 2003, Cognitive neuropsychology.

[17]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[18]  A. Caramazza,et al.  Concepts Are More than Percepts: The Case of Action Verbs , 2008, The Journal of Neuroscience.

[19]  D. Perani,et al.  The neural correlates of verb and noun processing. A PET study. , 1999, Brain : a journal of neurology.

[20]  Maryellen C. MacDonald,et al.  Context-dependent interpretation of words: Evidence for interactive neural processes , 2007, NeuroImage.

[21]  G. Orban,et al.  Comparative mapping of higher visual areas in monkeys and humans , 2004, Trends in Cognitive Sciences.

[22]  G. Altmann Language-mediated eye movements in the absence of a visual world: the ‘blank screen paradigm’ , 2004, Cognition.

[23]  N. Sadato,et al.  Neural representation of animacy in the early visual areas: A functional MRI study , 2009, Brain Research Bulletin.

[24]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.

[25]  Karen Emmorey,et al.  Modulation of BOLD Response in Motion-sensitive Lateral Temporal Cortex by Real and Fictive Motion Sentences , 2010, Journal of Cognitive Neuroscience.

[26]  Page Widick,et al.  Functional–anatomical organization of predicate metaphor processing , 2008, Brain and Language.

[27]  N. Kanwisher,et al.  Activation in Human MT/MST by Static Images with Implied Motion , 2000, Journal of Cognitive Neuroscience.

[28]  R. Gómez,et al.  Infant artificial language learning and language acquisition , 2000, Trends in Cognitive Sciences.

[29]  Lorraine K. Tyler,et al.  Objects and their actions: evidence for a neurally distributed semantic system , 2003, NeuroImage.

[30]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[31]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[32]  Ravi S. Menon,et al.  Distinguishing subregions of the human MT+ complex using visual fields and pursuit eye movements. , 2001, Journal of neurophysiology.

[33]  G. Altmann,et al.  Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation , 2009, Cognition.

[34]  Rolf A. Zwaan,et al.  Seeing, acting, understanding: motor resonance in language comprehension. , 2006, Journal of experimental psychology. General.

[35]  Yishi Jin,et al.  Getting a GRIP on Liprins , 2002, Neuron.

[36]  Katie L. McMahon,et al.  Action word meaning representations in cytoarchitectonically defined primary and premotor cortices , 2008, NeuroImage.

[37]  James K. Kroger,et al.  Cross-modal and cross-temporal association in neurons of frontal cortex , 2000, Nature.

[38]  Jeffery A. Jones,et al.  Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect , 2003, Neuroreport.

[39]  L. Tyler,et al.  Modulation of motor and premotor cortices by actions, action words and action sentences , 2009, Neuropsychologia.

[40]  Karl J. Friston,et al.  Activation in Posterior Superior Temporal Sulcus Parallels Parameter Inducing the Percept of Animacy , 2005, Neuron.

[41]  Rolf A. Zwaan,et al.  Constructing Multidimensional Situation Models During Reading , 1998 .

[42]  J. Fuster The Prefrontal Cortex—An Update Time Is of the Essence , 2001, Neuron.

[43]  Angela D. Friederici,et al.  Top-Down and Bottom-Up Contributions to Understanding Sentences Describing Objects in Motion , 2010, Front. Psychology.

[44]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[45]  Matthew W. Crocker,et al.  The influence of recent scene events on spoken comprehension: Evidence from eye movements , 2007 .

[46]  Antonino Vallesi,et al.  Effects of TMS on Different Stages of Motor and Non-Motor Verb Processing in the Primary Motor Cortex , 2009, PloS one.

[47]  John C. Trueswell,et al.  The role of thematic structures in interpretation and parsing. , 1989 .

[48]  Daniel C. Richardson,et al.  Representation, space and Hollywood Squares: looking at things that aren't there anymore , 2000, Cognition.

[49]  S Zeki,et al.  Going beyond the information given: the relation of illusory visual motion to brain activity , 1993, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[50]  C. Frith,et al.  Development and neurophysiology of mentalizing. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[51]  A. Dale,et al.  Visual motion aftereffect in human cortical area MT revealed by functional magnetic resonance imaging , 1995, Nature.

[52]  H Suzuki,et al.  Human cortical area responding to stimuli in apparent motion , 1997, Neuroreport.

[53]  Allen Ardestani,et al.  Distributed and associative working memory. , 2007, Cerebral cortex.

[54]  Christopher T. Kello,et al.  Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[55]  Michael P. Kaschak,et al.  Perception of motion affects language processing , 2005, Cognition.

[56]  E. Bullmore,et al.  The functional neuroanatomy of implicit-motion perception or ‘representational momentum’ , 2000, Current Biology.

[57]  Alex Martin,et al.  Semantic memory and the brain: structure and processes , 2001, Current Opinion in Neurobiology.

[58]  D. G. Albrecht,et al.  Spikes versus BOLD: what does neuroimaging tell us about neuronal activity? , 2000, Nature Neuroscience.

[59]  Abraham Z Snyder,et al.  Reliability of functional localization using fMRI , 2003, NeuroImage.

[60]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[61]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[62]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[63]  M. Brett,et al.  Actions Speak Louder Than Functions: The Importance of Manipulability and Action in Tool Representation , 2003, Journal of Cognitive Neuroscience.

[64]  Brandon R. Schrand Works Cited , 1991, Bach's Cycle, Mozart's Arrow.

[65]  J. Fuster,et al.  From perception to action: temporal integrative functions of prefrontal and parietal neurons. , 1999, Cerebral cortex.

[66]  Robert Davies,et al.  Neural Correlates of Abstract Verb Processing , 2011, Journal of Cognitive Neuroscience.

[67]  Olaf Hauk,et al.  Experience-dependent Plasticity of Conceptual Representations in Human Sensory-Motor Areas , 2007, Journal of Cognitive Neuroscience.

[68]  I. Johnsrude,et al.  The problem of functional localization in the human brain , 2002, Nature Reviews Neuroscience.

[69]  J. Haxby,et al.  fMRI Responses to Video and Point-Light Displays of Moving Humans and Manipulable Objects , 2003, Journal of Cognitive Neuroscience.

[70]  A. Vanlierde,et al.  Specific activation of the V5 brain area by auditory motion processing: an fMRI study. , 2005, Brain research. Cognitive brain research.

[71]  Bradford Z. Mahon,et al.  A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content , 2008, Journal of Physiology-Paris.

[72]  N Mai,et al.  Disturbance of movement vision after bilateral posterior brain damage. Further evidence and follow up observations. , 1991, Brain : a journal of neurology.

[73]  R. Andersen,et al.  Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[74]  L. Barsalou Situated simulation in the human conceptual system , 2003 .

[75]  J. Kable,et al.  Neural Substrates of Action Event Knowledge , 2002, Journal of Cognitive Neuroscience.

[76]  Alan C. Evans,et al.  A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning. , 2000, Cerebral cortex.

[77]  B. Rossion,et al.  Fixation Patterns During Recognition of Personally Familiar and Unfamiliar Faces , 2010, Front. Psychology.

[78]  Karl J. Friston,et al.  Action selectivity in parietal and temporal cortex. , 2005, Brain research. Cognitive brain research.

[79]  Richard S. J. Frackowiak,et al.  Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. , 1993, Cerebral cortex.

[80]  Silvia P. Gennari,et al.  Motion events in language and cognition , 2002, Cognition.

[81]  A. Damasio,et al.  Neural Correlates of Naming Actions and of Naming Spatial Relations , 2001, NeuroImage.

[82]  W. Singer,et al.  The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery , 1998, The European journal of neuroscience.

[83]  Rolf A. Zwaan,et al.  Language Comprehenders Mentally Represent the Shapes of Objects , 2002, Psychological science.

[84]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[85]  John C. Trueswell,et al.  Chapter 7 – Sentence Comprehension , 1995 .

[86]  A. Meltzoff,et al.  The detection of contingency and animacy from simple animations in the human brain. , 2003, Cerebral cortex.

[87]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .