Iconic gestures prime related concepts: An ERP study

To assess priming by iconic gestures, we recorded EEG (at 29 scalp sites) in two experiments while adults watched short, soundless videos of spontaneously produced, cospeech iconic gestures followed by related or unrelated probe words. In Experiment 1, participants classified the relatedness between gestures and words. In Experiment 2, they attended to stimuli, and performed an incidental recognition memory test on words presented during the EEG recording session. Event-related potentials (ERPs) time-locked to the onset of probe words were measured, along with response latencies and word recognition rates. Although word relatedness did not affect reaction times or recognition rates, contextually related probe words elicited less-negative ERPs than did unrelated ones between 300 and 500 msec after stimulus onset (N400) in both experiments. These findings demonstrate sensitivity to semantic relations between iconic gestures and words in brain activity engendered during word comprehension.

[1]  M. Kutas,et al.  Event-related brain potentials to semantically inappropriate and surprisingly large words , 1980, Biological Psychology.

[2]  M. Kutas,et al.  Event-related potential asymmetries during the reading of sentences. , 1988, Electroencephalography and clinical neurophysiology.

[3]  Heather Shovelton,et al.  An experimental investigation of some properties of individual iconic gestures that mediate their communicative power. , 2002, British journal of psychology.

[4]  Susan Goldin-Meadow,et al.  Gestures convey substantive information about a child’s thoughts to ordinary listeners , 1999 .

[5]  M. Pratarelli,et al.  Semantic Processing of Pictures and Spoken Words: Evidence from Event-Related Brain Potentials , 1994, Brain and Cognition.

[6]  T. Carr,et al.  Words, pictures, and priming: on semantic activation, conscious identification, and the automaticity of information processing. , 1982, Journal of experimental psychology. Human perception and performance.

[7]  G. Comi,et al.  IFCN standards for digital recording of clinical EEG. The International Federation of Clinical Neurophysiology. , 1998, Electroencephalography and clinical neurophysiology. Supplement.

[8]  Ying Choon Wu,et al.  Meaningful gestures: electrophysiological indices of iconic gesture comprehension. , 2005, Psychophysiology.

[9]  U. Bellugi,et al.  Neural Systems Mediating American Sign Language: Effects of Sensory Experience and Age of Acquisition , 1997, Brain and Language.

[10]  J P Rauschecker,et al.  Hemispheric specialization for English and ASL: left invariance‐right variability , 1998, Neuroreport.

[11]  M. Kutas,et al.  Interactions between sentence context and word frequencyinevent-related brainpotentials , 1990, Memory & cognition.

[12]  E. Bates,et al.  Meaning in gestures : What event-related potentials reveal about processes underlying the comprehension of iconic gestures , 2008 .

[13]  D. Hines.  Effect of Masked Picture Primes on Semantic Priming for Easy- and Difficult-to-Name Words , 1993 .

[14]  D. McNeill.  Hand and Mind , 1995 .

[15]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[16]  D. Deacon,et al.  Event-related potential indices of semantic priming using masked and unmasked words: evidence that the N400 does not reflect a post-lexical process. , 2000, Brain research. Cognitive brain research.

[17]  E. Vogel,et al.  Word meanings can be accessed but not reported during the attentional blink , 1996, Nature.

[18]  S. Kelly,et al.  Neural correlates of bimodal speech and gesture comprehension , 2004, Brain and Language.

[19]  Heather Shovelton,et al.  Mapping the Range of Information Contained in the Iconic Hand Gestures that Accompany Spontaneous Speech , 1999 .

[20]  G. Comi,et al.  IFCN standards for digital recording of clinical EEG. International Federation of Clinical Neurophysiology. , 1998, Electroencephalography and clinical neurophysiology.

[21]  E. Plante,et al.  Time course of word identification and semantic integration in spoken language. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[22]  A. A. Wijers,et al.  Visual semantic features are activated during the processing of concrete words: event-related potential evidence for perceptual semantic priming. , 2000, Brain research. Cognitive brain research.

[23]  A. Kok.  On the utility of P3 amplitude as a measure of processing capacity. , 2001, Psychophysiology.

[24]  M. Kutas,et al.  The Search for Common Sense: An Electrophysiological Study of the Comprehension of Words and Pictures in Reading , 1996, Journal of Cognitive Neuroscience.

[25]  D. McNeill,et al.  Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information , 1998 .

[26]  W. Rogers,et al.  THE CONTRIBUTION OF KINESIC ILLUSTRATORS TOWARD THE COMPREHENSION OF VERBAL BEHAVIOR WITHIN UTTERANCES , 1978 .

[27]  Phillip J Holcomb,et al.  Semantic integration in videos of real-world events: an electrophysiological investigation. , 2003, Psychophysiology.

[28]  Susan Goldin-Meadow,et al.  Assessing knowledge conveyed in gesture : Do teachers have the upper hand ? , 1997 .

[29]  R. Krauss,et al.  Do conversational hand gestures communicate? , 1991, Journal of personality and social psychology.

[30]  H. Neville,et al.  Language and , 2019, Adventure Diffusion.

[31]  C. Braun,et al.  Speech rate as a sticky switch: A multiple lesion case analysis of mutism and hyperlalia , 2004, Brain and Language.

[32]  J. Raaijmakers,et al.  Does pizza prime coin? Perceptual priming in lexical decision and pronunciation. , 1998 .

[33]  P. Holcomb,et al.  An electrophysiological investigation of semantic priming with pictures of real objects. , 1999, Psychophysiology.

[34]  C Van Petten,et al.  Time course of word identification and semantic integration in spoken language. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[35]  M. Rugg,et al.  Event-related potentials and the semantic matching of pictures , 1990, Brain and Cognition.

[36]  Patric Bach,et al.  Communicating hands: ERPs elicited by meaningful symbolic hand postures , 2004, Neuroscience Letters.

[37]  E. Wagenmakers,et al.  A Model-Averaging Approach to Replication : The Case of p rep , 2009 .

[38]  R. Klatzky,et al.  Teachers’ gestures facilitate students’ learning: A lesson in symmetry , 2003 .

[39]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[40]  M. Vanderwart,et al.  Priming by pictures in lexical decision , 1984 .