EFFECTS OF DIFFERENT TYPES OF FACE COVERINGS ON SPEECH ACOUSTICS AND INTELLIGIBILITY

This paper reports the results of two experiments investigating the effects on speech acoustics and intelligibility of a number of different types of forensically-relevant fabric mouth and face coverings, including the niqāb (full-face Muslim veil), balaclava, and surgical mask. For the perceptual (intelligibility) experiment, subjects were presented with two types of speech stimuli, ‘bimodal’ and ‘unimodal’, and asked to write down what they heard. Four facial guises were used (niqāb, balaclava, surgical mask, no covering). In the bimodal condition (video + audio), subjects saw and heard video recordings of actors reading target words embedded in a standardised carrier sentence. In the unimodal (audio only) condition, subjects heard just the soundtrack of the same video recordings, i.e., no visual image was present. It was found in the perceptual test that the subjects could in all four guise conditions correctly identify target words with a high degree of reliability, and that a small number of confusion types accounted for the majority of the errors. For the second (acoustic) experiment, the objective was to assess the sound transmission loss characteristics of the fabrics from which these and other face coverings are composed. This experiment showed that transmission loss was negligible for all but one of the fabrics, suggesting that speech intelligibility problems created when mouth and face coverings are worn by speakers must derive principally from the reduction in visual information available to listeners and/or from the auditory consequences of interference with speech articulation caused by the face coverings, rather than from transmission loss of the fabrics themselves.

[1]  Peter F. Assmann,et al.  Effects of visual gender and frequency shifts on vowel category judgments , 2004 .

[2]  Lawrence Brancazio,et al.  Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect , 2005, Perception & psychophysics.

[3]  Q. Summerfield Some preliminaries to a comprehensive account of audio-visual speech perception. , 1987 .

[4]  Jean‐Pierre A. Radley,et al.  Acoustic Properties of Stop Consonants , 1957 .

[5]  Joanne L. Miller,et al.  Speech Perception , 1990, Springer Handbook of Auditory Research.

[6]  R. Remez,et al.  Perceptual Organization of Speech , 2008, The Handbook of Speech Perception.

[7]  Ruth Campbell,et al.  The processing of audio-visual speech: empirical and neural bases , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[8]  Ruben van de Vijver,et al.  Pisoni, D., Remez, R. (eds.), The handbook of speech perception; Oxford, Blackwell, 2005 , 2009 .

[9]  G. A. Miller,et al.  An Analysis of Perceptual Confusions Among Some English Consonants , 1955 .

[10]  Z S Bond,et al.  Acoustic-phonetic characteristics of speech produced in noise and while wearing an oxygen mask. , 1989, The Journal of the Acoustical Society of America.

[11]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[12]  M. E. Nute,et al.  63—THE EFFECT OF FABRIC PARAMETERS ON SOUND-TRANSMISSION LOSS , 1973 .

[13]  G. McConkie,et al.  Attention to facial regions in segmental and prosodic visual speech perception tasks. , 1999, Journal of speech, language, and hearing research : JSLHR.

[14]  N. P. Erber,et al.  Auditory-visual perception of speech with reduced optical clarity. , 1979, Journal of speech and hearing research.

[15]  David Coniam,et al.  The Impact of Wearing a Face Mask in a High-Stakes Oral Examination: An Exploratory Post-SARS Study in Hong Kong , 2005 .

[16]  Eric Vatikiotis-Bateson,et al.  The moving face during speech communication , 1998 .

[17]  Elizabeth A. Strand,et al.  Auditory–visual integration of talker gender in vowel perception , 1999 .

[18]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[19]  J. C. Cotton NORMAL "VISUAL HEARING". , 1935, Science.

[20]  K. Green The Use of Auditory and Visual Information in Phonetic Perception , 1996 .

[21]  A. F.,et al.  Architectural Acoustics , 1933, Nature.

[22]  R. Campbell,et al.  Hearing by eye 2 : advances in the psychology of speechreading and auditory-visual speech , 1997 .

[23]  J Bishop,et al.  Near-field speech intelligibility in chemical-biological warfare masks. , 1999, Military medicine.

[24]  John Vanderkooy,et al.  Transfer-Function Measurement with Maximum-Length Sequences , 1989 .

[25]  Jeffery A. Jones,et al.  Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect , 2003, Neuroreport.

[26]  D. Massaro Speech Perception By Ear and Eye: A Paradigm for Psychological Inquiry , 1989 .

[27]  S. Adanur Handbook of Weaving , 2000 .

[28]  Keith Johnson,et al.  Gradient and Visual Speaker Normalization in the Perception of Fricatives , 1996, KONVENS.

[29]  Jennifer Hay,et al.  Factors influencing speech perception in the context of a merger-in-progress , 2006, J. Phonetics.