Effect of Machine Translation in Interlingual Conversation: Lessons from a Formative Study

Language barrier is the primary challenge for effective cross-lingual conversations. Spoken language translation (SLT) is perceived as a cost-effective alternative to less affordable human interpreters, but little research has studied how people interact with such technology. Using a prototype translator application, we performed a formative evaluation to elicit how people interact with the technology and adapt their conversation style. We conducted two sets of studies with a total of 23 pairs (46 participants). Participants worked on storytelling tasks to simulate natural conversations with 3 different interface settings. Our findings show that collocutors naturally adapt their style of speech production and comprehension to compensate for inadequacies in SLT. We conclude the paper with design guidelines that emerged from the analysis.

[1]  Li Deng,et al.  Challenges in adopting speech recognition , 2004, CACM.

[2]  Ebru Arisoy,et al.  Morph-based speech recognition and modeling of out-of-vocabulary words across languages , 2007, TSLP.

[3]  James Glass,et al.  Modelling out-of-vocabulary words for robust speech recognition , 2002 .

[4]  Wei Chen,et al.  Active error detection and resolution for speech-to-speech translation , 2012, IWSLT.

[5]  Bin Xu,et al.  How beliefs about the presence of machine translation impact multilingual collaborations , 2014, CSCW.

[6]  Alexander H. Waibel,et al.  Simultaneous translation of lectures and speeches , 2007, Machine Translation.

[7]  Toru Ishida,et al.  Effects of machine translation on collaborative work , 2006, CSCW '06.

[8]  Alexander H. Waibel,et al.  End-to-End Evaluation in Simultaneous Translation , 2009, EACL.

[9]  Li Deng,et al.  Speech-Centric Information Processing: An Optimization-Oriented Approach , 2013, Proceedings of the IEEE.

[10]  Fabio Pianesi,et al.  The added value of multimodality in the NESPOLE! speech-to-speech translation system: an experimental study , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[11]  B. Berg.  Qualitative Research Methods for the Social Sciences , 1989 .

[12]  Naoko Morita,et al.  Negotiating Participation and Identity in Second Language Academic Communities. , 2004 .

[13]  Daniel Gile,et al.  Chapter 1. Theoretical components in interpreter and translator training , 2009 .

[14]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[15]  Susan R. Fussell,et al.  Machine translation vs. common language: effects on idea exchange in cross-lingual groups , 2013, CSCW.

[16]  Emine Kale,et al.  Language barriers and the use of interpreters in the public health services. A questionnaire-based survey. , 2010, Patient education and counseling.

[17]  Hideaki Kuzuoka,et al.  Difficulties in establishing common ground in multiparty groups using machine translation , 2009, CHI.

[18]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[19]  Alex Waibel,et al.  JANUS: a speech-to-speech translation system using connectionist and symbolic processing strategies , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[20]  Michael Picheny,et al.  Effects of real-time transcription on non-native speaker's comprehension in computer-mediated communications , 2009, CHI.

[21]  Satoshi Nakamura,et al.  The ATR Multilingual Speech-to-Speech Translation System , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[22]  Alexander G. Hauptmann,et al.  Connectionist and Symbolic Processing in Speech-to-Speech Translation: The JANUS System , 1991 .

[23]  A. Singer,et al.  Contemporary Immigrant Gateways in Historical Perspective , 2013, Daedalus.

[24]  Toshiyuki Takezawa,et al.  A new evaluation method for speech translation systems and a case study on ATR-MATRIX from Japanese to English , 1999, MTSUMMIT.

[25]  Anne-Wil Harzing,et al.  Language management in multinational companies , 2003 .

[26]  James R. Glass,et al.  Modeling out-of-vocabulary words for robust speech recognition , 2000, INTERSPEECH.

[27]  Shamsi T. Iqbal,et al.  Sharing a driver's context with a caller via continuous audio cues to increase awareness about driver state. , 2014, Journal of experimental psychology. Applied.

[28]  Susan R. Fussell,et al.  Effects of public vs. private automated transcripts on multiparty communication between native and non-native english speakers , 2014, CHI.