Neurocognition of speech-specific audiovisual perception

We have studied neurocognitive mechanisms of audiovisual perception of non-meaningful speech. We demonstrate different event-related brain potential (ERP) differences to non-speech and speech audiovisual stimuli. Our imaging studies show that viewing and listening to speech activate overlapping areas in the frontal cortex, which is also activated during speech production. We also demonstrate that subjects have to expect the auditory stimuli to be speech to integrate auditory and visual speech.

[1]  P. McGuire,et al.  Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). , 2001, Brain research. Cognitive brain research.

[2]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[3]  T. Paus,et al.  Seeing and hearing speech excites the motor system involved in speech production , 2003, Neuropsychologia.

[4]  M. Sams,et al.  Time course of multisensory interactions during audiovisual speech perception in humans: a magnetoencephalographic study , 2004, Neuroscience Letters.

[5]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[6]  Christian Kaufmann,et al.  Human prefrontal and sensory cortical activity during divided attention tasks , 2003, Human brain mapping.

[7]  R. Hari,et al.  Viewing Lip Forms Cortical Dynamics , 2002, Neuron.

[8]  Mikko Sams,et al.  Processing of audiovisual speech in Broca's area , 2005, NeuroImage.

[9]  Alan C. Evans,et al.  Lateralization of phonetic and pitch discrimination in speech processing. , 1992, Science.

[10]  R. Hari,et al.  Seeing speech: visual information from lip movements modifies activity in the human auditory cortex , 1991, Neuroscience Letters.

[11]  G. Rizzolatti,et al.  Hearing Sounds, Understanding Actions: Action Representation in Mirror Neurons , 2002, Science.

[12]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[13]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[14]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[15]  Klucharev Vasily,et al.  Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. , 2003 .

[16]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.