How Non-native Speakers Perceive Listening Comprehension Problems: Implications for Adaptive Support Technologies

Previous studies have suggested many technologies to support non-native speaker comprehension in real-time communication. However, such technologies may impose an extra burden on non-native speakers (NNSs) if they do not match their current needs. To design a system that adapts to the changing needs of NNSs, we need to understand the types of problems NNSs face and how these problems are perceived by them. To explore such issues, we conducted a laboratory experiment with 40 NNSs (and 20 native speakers as a control group) who engaged in a listening task. During the task, the participants pressed a button whenever they encountered a comprehension problem. Next they explained each problem, the point at which they recognized the problem, and for how long it persisted. Our analysis identified twelve types of listening comprehension problems, which we further classified into three patterns based on their persistence and the time taken to perceive them. Our findings have implications for designing adaptive technologies to support listening comprehension of NNSs in real-time communication.

[1]  Michael Picheny,et al.  Effects of automated transcription quality on non-native speakers' comprehension in real-time computer-mediated communication , 2010, CHI.

[2]  A. Zekveld,et al.  Pupil Response as an Indication of Effortful Listening: The Influence of Sentence Intelligibility , 2010, Ear and hearing.

[3]  Wolfgang Wahlster,et al.  Verbmobil: Foundations of Speech-to-Speech Translation , 2000, Artificial Intelligence.

[4]  Akiko Noda,et al.  A Temporary Decline of Thinking Ability During Foreign Language Processing , 1993 .

[5]  Susan R. Fussell,et al.  Improving Multilingual Collaboration by Displaying How Non-native Speakers Use Automated Transcripts and Bilingual Dictionaries , 2015, CHI.

[6]  Jos J. A. Van Berkum,et al.  Does working memory capacity affect the ability to predict upcoming words in discourse? , 2009, Brain Research.

[7]  Pamela J. Hinds,et al.  Walking Through Jelly: Language Proficiency, Emotions, and Disrupted Collaboration in Global Work , 2009 .

[8]  Steve Whittaker,et al.  Time-Compressing Speech: ASR Transcripts Are an Effective Way to Support Gist Extraction , 2008, MLMI.

[9]  Kristina Höök,et al.  Steps to take before intelligent user interfaces become real , 2000, Interact. Comput..

[10]  Luisa C. Perez.  Foreign Language Productivity in Synchronous versus Asynchronous Computer-Mediated Communication , 2013 .

[11]  Denice Welch,et al.  Language: The forgotten factor in multinational management , 1997 .

[12]  Ghazal Lotfi.  A Questionnaire of Beliefs on English Language Listening Comprehension Problems: Development and Validation , 2012 .

[13]  T. Cox,et al.  Managing cultural diversity: implications for organizational competitiveness , 1991 .

[14]  Lin Yao,et al.  Effects of Automated Transcription Delay on Non-native Speakers' Comprehension in Real-Time Computer-Mediated Communication , 2011, INTERACT.

[15]  Larry Vandergrift.  Recent developments in second and foreign language listening comprehension research , 2007, Language Teaching.

[16]  Tammo Houtgast,et al.  The Influence of Age, Hearing, and Working Memory on the Speech Comprehension Benefit Derived from an Automatic Speech Recognition System , 2009, Ear and hearing.

[17]  Vaiva Kalnikaité,et al.  Markup as you talk: establishing effective memory cues while still contributing to a meeting , 2012, CSCW.

[18]  C. Goh.  Exploring listening comprehension tactics and their interaction patterns , 2002 .

[19]  Michael C. Dorneich,et al.  Towards a Characterization of Adaptive Systems: a Framework for Researchers and System Designers , 2017 .

[20]  Satoshi Nakamura,et al.  The ATR Multilingual Speech-to-Speech Translation System , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  S. Ross,et al.  What makes listening difficult? Factors affecting second language listening comprehension , 2010 .

[22]  Toru Ishida,et al.  Language grid: an infrastructure for intercultural collaboration , 2006, International Symposium on Applications and the Internet (SAINT'06).

[23]  Naomi Yamashita,et al.  Do automated transcripts help non-native speakers catch up on missed conversation in audio conferences? , 2014, CABS '14.

[24]  J. Rubin.  A Review of Second Language Listening Comprehension Research , 1994 .

[25]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[26]  Toru Ishida,et al.  Lost in transmittance: how transmission lag enhances and deteriorates multilingual collaboration , 2013, CSCW.

[27]  Susan R. Fussell,et al.  Effects of public vs. private automated transcripts on multiparty communication between native and non-native english speakers , 2014, CHI.

[28]  Christine C. M. Goh,et al.  How Much Do Learners Know about the Factors That Influence Their Listening Comprehension , 1999 .

[29]  C. Goh.  A cognitive perspective on language learners' listening comprehension problems , 2000 .

[30]  C. Van Petten,et al.  Prediction during language comprehension: benefits, costs, and ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[31]  Anne-Wil Harzing,et al.  Language management in multinational companies , 2003 .

[32]  M. Pickering,et al.  Do people use language production to make predictions during comprehension? , 2007, Trends in Cognitive Sciences.