Semantic processing of mathematical gestures

OBJECTIVE To examine whether or not university mathematics students semantically process gestures depicting mathematical functions (mathematical gestures) similarly to the way they process action gestures and sentences. Semantic processing was indexed by the N400 effect. RESULTS The N400 effect elicited by words primed with mathematical gestures (e.g. "converging" and "decreasing") was the same in amplitude, latency and topography as that elicited by words primed with action gestures (e.g. drive and lift), and that for terminal words of sentences. SIGNIFICANCE AND CONCLUSION Findings provide a within-subject demonstration that the topographies of the gesture N400 effect for both action and mathematical words are indistinguishable from that of the standard language N400 effect. This suggests that mathematical function words are processed by the general language semantic system and do not appear to involve areas involved in other mathematical concepts (e.g. numerosity).

[1]  Anne L. Foundas,et al.  Speech and gesture are mediated by independent systems , 2005, Behavioral and Brain Sciences.

[2]  Blake W. Johnson,et al.  High-density mapping in an N400 paradigm: evidence for bilateral temporal lobe generators , 2000, Clinical Neurophysiology.

[3]  Volkmar Glauche,et al.  The human action recognition system and its relationship to Broca’s area: an fMRI study , 2003, NeuroImage.

[4]  M Sarfarazi,et al.  Visual event related potentials modulated by contextually relevant and irrelevant olfactory primes. , 1999, Chemical senses.

[5]  M. Sams,et al.  Primary auditory cortex activation by visual speech: an fMRI study at 3 T , 2005, Neuroreport.

[6]  M. Arbib.  From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics , 2005, Behavioral and Brain Sciences.

[7]  John R. Anderson,et al.  Endogenous Control and Task Representation: An fMRI Study in Algebraic Problem-solving , 2008, Journal of Cognitive Neuroscience.

[8]  S. Dehaene,et al.  THREE PARIETAL CIRCUITS FOR NUMBER PROCESSING , 2003, Cognitive neuropsychology.

[9]  Lyn D. English,et al.  Handbook of International Research in Mathematics Education , 2002 .

[10]  C. D. Binnie,et al.  A simple algorithm for a digital three-pole Butterworth filter of arbitrary cut-off frequency: application to digital electroencephalography , 2000, Journal of Neuroscience Methods.

[11]  Roel M. Willems,et al.  When language meets action: the neural integration of gesture and speech. , 2007, Cerebral cortex.

[12]  Willem J. M. Levelt,et al.  Pointing and voicing in deictic expressions , 1985 .

[13]  Stanislas Dehaene,et al.  Arithmetic and the Brain This Review Comes from a Themed Issue on Cognitive Neuroscience Edited the Intraparietal Sulcus and Number Sense Number Sense in the Animal Brain , 2022 .

[14]  N. Reissland.  From Hand to Mouth: The Origins of Language , 2006 .

[15]  D. Lehmann,et al.  Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. , 1994, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[16]  John R. Anderson,et al.  The roles of prefrontal and posterior parietal cortex in algebra problem solving: A case of using cognitive modeling to inform neuroimaging data , 2007, NeuroImage.

[17]  Ying Choon Wu,et al.  Meaningful gestures: electrophysiological indices of iconic gesture comprehension. , 2005, Psychophysiology.

[18]  John R. Anderson,et al.  An information-processing model of the BOLD response in symbol manipulation tasks , 2003, Psychonomic bulletin & review.

[19]  Rochel Gelman,et al.  Language and Conceptual Development series Number and language : how are they related ? , 2004 .

[20]  A. Caramazza,et al.  Concepts Are More than Percepts: The Case of Action Verbs , 2008, The Journal of Neuroscience.

[21]  Sotaro Kita,et al.  On-line Integration of Semantic Information from Speech and Gesture: Insights from Event-related Brain Potentials , 2007, Journal of Cognitive Neuroscience.

[22]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[23]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[24]  Ying Choon Wu,et al.  How iconic gestures enhance communication: An ERP study , 2007, Brain and Language.

[25]  U. Hadar,et al.  Gesture and the Processing of Speech: Neuropsychological Evidence , 1998, Brain and Language.

[26]  Ornella Robutti,et al.  Gestures as semiotic resources in the mathematics classroom , 2009 .

[27]  Andreas Hennenlotter,et al.  Neural correlates of the processing of co-speech gestures , 2008, NeuroImage.

[28]  C. C. Wood,et al.  Scalp distributions of event-related potentials: an ambiguity associated with analysis of variance models. , 1985, Electroencephalography and clinical neurophysiology.

[29]  Susan Goldin.  Hearing gesture : how our hands help us think , 2003 .

[30]  P. Holcomb,et al.  An electrophysiological investigation of semantic priming with pictures of real objects. , 1999, Psychophysiology.

[31]  M. Rugg,et al.  Event-related potentials and the semantic matching of pictures , 1990, Brain and Cognition.

[32]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[33]  Maurizio Gentilucci,et al.  Please Scroll down for Article the Quarterly Journal of Experimental Psychology Spoken Language and Arm Gestures Are Controlled by the Same Motor Control System , 2022 .

[34]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[35]  J. Connolly,et al.  Event-Related Potential Components Reflect Phonological and Semantic Processing of the Terminal Word of Spoken Sentences , 1994, Journal of Cognitive Neuroscience.

[36]  John Polich,et al.  N400s from sentences, semantic categories, number and letter strings? , 1985 .

[37]  Ferdinando Arzarello,et al.  Introduction: beyond words , 2009 .

[38]  D. Lehmann,et al.  Reference-free identification of components of checkerboard-evoked multichannel potential fields. , 1980, Electroencephalography and clinical neurophysiology.

[39]  Blake W. Johnson,et al.  Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts , 2002, Clinical Neurophysiology.

[40]  D. Tucker.  Spatial sampling of head electrical fields: the geodesic sensor net. , 1993, Electroencephalography and clinical neurophysiology.

[41]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[42]  R. C. Oldfield.  THE ASSESSMENT AND ANALYSIS OF HANDEDNESS , 1971 .

[43]  John F. Connolly,et al.  Assessment of children's receptive vocabulary using event-related brain potentials: Development of a clinically valid test , 1995 .

[44]  M. Kutas,et al.  An Event-Related Potential (ERP) Analysis of Semantic Congruity and Repetition Effects in Sentences , 1992, Journal of Cognitive Neuroscience.

[45]  Blake W. Johnson,et al.  One good turn deserves another: an event-related brain potential study of rotated mirror–normal letter discriminations , 2004, Neuropsychologia.

[46]  James Bartolotti,et al.  An intentional stance modulates the integration of gesture and speech during comprehension , 2007, Brain and Language.

[47]  Thomas C. Gunter,et al.  The Role of Iconic Gestures in Speech Disambiguation: ERP Evidence , 2007, Journal of Cognitive Neuroscience.