Deaf and hard-of-hearing users' preferences for hearing speakers' behavior during technology-mediated in-person and remote conversations

Various technologies mediate synchronous audio-visual one-on-one communication (SAVOC) between Deaf and Hard-of-Hearing (DHH) and hearing colleagues, including automatic-captioning smartphone apps for in-person settings, or text-chat features of videoconferencing software in remote settings. Speech and non-verbal behaviors of hearing speakers, e.g. speaking too quietly, can make SAVOC difficult for DHH users, but prior work had not examined technology-mediated contexts. In an in-person study (N=20) with an automatic captioning smartphone app, variations in a hearing actor's enunciation and intonation dynamics affected DHH users' satisfaction. In a remote study (N=23) using a videoconferencing platform with text chat, variations in speech rate, voice intensity, enunciation, intonation dynamics, and eye contact led to such differences. This work contributes empirical evidence that specific behaviors of hearing speakers affect the accessibility of technology-mediated SAVOC for DHH users, providing motivation for future work on detecting or encouraging useful communication behaviors among hearing individuals.

[1]  F J IJsseldijk,et al.  Speechreading performance under different conditions of video image, repetition, and speech rate. , 1992, Journal of Speech and Hearing Research.

[2]  Christian Vogler,et al.  Mixed local and remote participation in teleconferences from a deaf and hard of hearing perspective , 2013, ASSETS.

[3]  H. H. Clark,et al.  Speaking while monitoring addressees for understanding , 2004 .

[4]  J. Harris,et al.  The Uphill Struggle: Services for Deaf and hard of hearing people - issues of equality, participation and access , 2001 .

[5]  Carol Padden,et al.  Inside Deaf Culture , 2005 .

[6]  N I Durlach,et al.  Speaking clearly for the hard of hearing I: Intelligibility differences between clear and conversational speech. , 1985, Journal of speech and hearing research.

[7]  Amanda Stent,et al.  Adapting speaking after evidence of misrecognition: Local and global hyperarticulation , 2008, Speech Commun..

[8]  Matt Huenerfauth,et al.  Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers , 2018, ASSETS.

[9]  Anne Marie Piper,et al.  Supporting medical conversations between deaf and hearing individuals with tabletop displays , 2008, CSCW.

[10]  Matt Huenerfauth,et al.  Deaf and Hard of Hearing Individuals' Perceptions of Communication with Hearing Colleagues in Small Groups , 2016, ASSETS.

[11]  M. Picheny,et al.  Speaking clearly for the hard of hearing. II: Acoustic characteristics of clear and conversational speech. , 1986, Journal of speech and hearing research.

[12]  Bonnie O'Day,et al.  Communicating about Health Care: Observations from Persons Who Are Deaf or Hard of Hearing , 2004, Annals of Internal Medicine.

[13]  Jing Shen,et al.  On Dynamic Pitch Benefit for Speech Recognition in Speech Masker , 2018, Front. Psychol..

[14]  Nancy Tye-Murray,et al.  Foundations of Aural Rehabilitation: Children, Adults, and Their Family Members , 1998 .

[15]  Rein Ove Sikveland,et al.  How do we Speak to Foreigners? – Phonetic Analyses of Speech Communication between L1 and L2 Speakers of Norwegian , 2009 .

[16]  J. Lucas,et al.  Summary health statistics for U.S. adults: national health interview survey, 2012. , 2014, Vital and health statistics. Series 10, Data from the National Health Survey.

[17]  A Boothroyd,et al.  Speechreading enhancement by voice fundamental frequency: the effects of Fo contour distortions. , 1992, Journal of speech and hearing research.

[18]  Angela M. O'Donnell.  The Role of Peers and Group Learning , 2006 .

[19]  David R. Flatla,et al.  MirrorMirror: A Mobile Application to Improve Speechreading Acquisition , 2018, CHI.

[20]  S Oviatt,et al.  Modeling global and focal hyperarticulation during human-computer error resolution. , 1998, The Journal of the Acoustical Society of America.

[21]  G. Walter.  Deaf and Hard-of-Hearing Students in Transition: Demographics with an Emphasis on STEM Education , 2010 .

[22]  Lisa B. Elliot,et al.  User Experiences When Testing a Messaging App for Communication Between Individuals who are Hearing and Deaf or Hard of Hearing , 2017, ASSETS.

[23]  Jon E. Froehlich,et al.  Autoethnography of a Hard of Hearing Traveler , 2019, ASSETS.

[24]  N I Durlach,et al.  Speaking clearly for the hard of hearing. III: An attempt to determine the contribution of speaking rate to differences in intelligibility between clear and conversational speech. , 1989, Journal of speech and hearing research.

[25]  Lisa B. Elliot,et al.  Personal Perspectives on Using Automatic Speech Recognition to Facilitate Communication between Deaf Students and Hearing Customers , 2017, ASSETS.

[26]  P. Wong,et al.  Testing the Hyperarticulation and Prosodic Hypotheses of Child-Directed Speech: Insights From the Perceptual and Acoustic Characteristics of Child-Directed Cantonese Tones. , 2018, Journal of speech, language, and hearing research : JSLHR.

[27]  Jon Barker,et al.  The CHiME Challenges: Robust Speech Recognition in Everyday Environments , 2017, New Era for Robust Speech Recognition, Exploiting Deep Learning.

[28]  Maria Uther,et al.  Do you speak E-NG-L-I-SH? A comparison of foreigner- and infant-directed speech , 2007, Speech Commun..

[29]  Richard E. Ladner,et al.  Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students , 2016, ASSETS.

[30]  Matt Huenerfauth,et al.  Deaf Individuals' Views on Speaking Behaviors of Hearing Peers when Using an Automatic Captioning App , 2020, CHI Extended Abstracts.

[31]  Walter S. Lasecki,et al.  Accessibility Evaluation of Classroom Captions , 2014, ACM Trans. Access. Comput..

[32]  I. Leigh,et al.  Comparison of Face-to-Face and Video-Mediated Communication with Deaf Individuals: Implications for Telepsychotherapy , 2019 .

[33]  David B Pisoni,et al.  Some normative data on lip-reading skills (L). , 2011, The Journal of the Acoustical Society of America.

[34]  Larwan Berke,et al.  Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings , 2017, ASSETS.

[35]  Jeannine S. Schiller,et al.  Summary health statistics for U.S. adults: National Health Interview Survey, 2002. , 2004, Vital and health statistics. Series 10, Data from the National Health Survey.

[36]  Denis K Burnham,et al.  Computer- and human-directed speech before and after correction , 2010 .

[37]  William Lewis,et al.  Skype Translator: Breaking down language and hearing barriers. A behind the scenes look at near real-time speech translation , 2015, TC.