Asymmetries for the visual expression and perception of speech

This study explored asymmetries for movement, expression and perception of visual speech. Sixteen dextral models were videoed as they articulated: 'bat,' 'cat,' 'fat,' and 'sat.' Measurements revealed that the right side of the mouth was opened wider and for a longer period than the left. The asymmetry was accentuated at the beginning and ends of the vocalization and was attenuated for words where the lips did not articulate the first consonant. To measure asymmetries in expressivity, 20 dextral observers watched silent videos and reported what was said. The model's mouth was covered so that the left, right or both sides were visible. Fewer errors were made when the right mouth was visible compared to the left--suggesting that the right side is more visually expressive of speech. Investigation of asymmetries in perception using mirror-reversed clips revealed that participants did not preferentially attend to one side of the speaker's face. A correlational analysis revealed an association between movement and expressivity whereby a more motile right mouth led to stronger visual expressivity of the right mouth. The asymmetries are most likely driven by left hemisphere specialization for language, which causes a rightward motoric bias.

[1]  M Studdert-Kennedy,et al.  Infant intermodal speech perception is a left-hemisphere function. , 1983, Science.

[2]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[3]  Mineo Yoshino,et al.  Detecting hemifacial asymmetries in emotional expression with three–dimensional computerized image analysis , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[4]  Ronald L. Boring,et al.  The Distribution of Attention Across a Talker's Face , 2004 .

[5]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[6]  D. Perrett,et al.  Perceptual asymmetries in judgements of facial attractiveness, age, gender, speech and expression , 1997, Neuropsychologia.

[7]  R. Campbell,et al.  Asymmetries in moving faces. , 1982, British journal of psychology.

[8]  D W Massaro,et al.  Laterality in visual speech perception. , 1998, Journal of experimental psychology. Human perception and performance.

[9]  A A Montgomery,et al.  Auditory and visual contributions to the perception of consonants. , 1974, Journal of speech and hearing research.

[10]  Ruth Campbell,et al.  The lateralization of lip-reading: A second look , 1996, Neuropsychologia.

[11]  Laura Ann Petitto,et al.  Left Hemisphere Cerebral Specialization for Babies While Babbling , 2002, Science.

[12]  Melvyn A. Goodale,et al.  Oral asymmetries during verbal and non-verbal movements of the mouth , 1987, Neuropsychologia.

[13]  Mouth asymmetry during speech of epileptic patients who have undergone carotid amytal testing , 1990, Neuropsychologia.

[14]  E. Owens,et al.  Visemes observed by hearing-impaired and normal-hearing adult viewers. , 1985, Journal of speech and hearing research.

[15]  J. Driver.  Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading , 1996, Nature.

[16]  D. Spencer,et al.  Cerebral dominance for speech and handwriting of patients with cortical vascular malformations. , 1994, Journal of clinical and experimental neuropsychology.

[17]  T. Landis,et al.  Singing with and without words: hemispheric asymmetries in motor control. , 1994, Journal of clinical and experimental neuropsychology.

[18]  D. Moody,et al.  Neural lateralization of species-specific vocalizations by Japanese macaques (Macaca fuscata). , 1978, Science.

[19]  M. Funnell,et al.  Hemispheric contributions to the integration of visual and auditory information in speech perception , 1994, Perception & psychophysics.

[20]  L. Rogers,et al.  Lateralized use of the mouth in production of vocalizations by marmosets , 1998, Neuropsychologia.

[21]  O. Güntürkün,et al.  Sex differences in oral asymmetries during wordrepetition , 1998, Neuropsychologia.

[22]  Murray Alpert,et al.  Facial asymmetry while posing positive and negative emotions: Support for the right hemisphere hypothesis , 1988, Neuropsychologia.

[23]  R. Campbell.  The lateralization of lip-read sounds: A first look , 1986, Brain and Cognition.

[24]  Theodor Landis,et al.  ASYMMETRY IN MOUTH OPENING DURING DIFFERENT SPEECH TASKS , 1990 .

[25]  John L. Bradshaw,et al.  Read My Lips , 2004, Psychological science.

[26]  D. Livesay.  Right hand left hand , 1977 .