Investigating Ways of Interpretations of Artificial Subtle Expressions Among Different Languages: A Case of Comparison Among Japanese, German, Portuguese and Mandarin Chinese

Up until now, several studies have shown that a speech interface system giving verbal suggestions with beeping sounds that decrease in pitch conveyed a low system confidence level to users intuitively, and these beeping sounds were named “artificial subtle expressions” (ASEs). However, all participants in these studies were only Japanese, so if the participants’ mother tongue has different sensitivity to variations in pitch compared with Japanese, the interpretations of the ASEs might be different. We then investigated whether the ASEs are interpreted in the same way as with Japanese regardless of the users’ mother tongues; specifically we focused on three language categories in traditional phonological typology. We conducted a web-based experiment to investigate whether the ways speakers of German, Portuguese (stress accent language), Mandarin Chinese (tone language) and Japanese (pitch accent language) interpret the ASEs are different or not. The results of this experiment showed that the ways of interpreting did not differ, so this suggests that these ways are language-independent.

[1]  Katsuhito Sudoh,et al.  Incorporating discourse features into confidence scoring of intention recognition results in spoken dialogue systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[2]  Seiji Yamada,et al.  Proposing Artificial Subtle Expressions as an Intuitive Notification Methodology for Artificial Agents' Internal States , 2010 .

[3]  Seiji Yamada,et al.  Artificial subtle expressions: intuitive notification methodology of artifacts , 2010, CHI.

[4]  H. Cai,et al.  Tuning Trust Using Cognitive Cues for Better Human-Machine Collaboration , 2010 .

[5]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[6]  W. Keith Edwards,et al.  Intelligibility and Accountability: Human Considerations in Context-Aware Systems , 2001, Hum. Comput. Interact..

[7]  James D. McCawley,et al.  What Is a Tone Language , 1978 .

[8]  Stuart E. Middleton,et al.  Capturing knowledge of user preferences: ontologies in recommender systems , 2001, K-CAP '01.

[9]  Eric Horvitz,et al.  Display of Information for Time-Critical Decision Making , 1995, UAI.

[10]  Bernt Schiele,et al.  Towards improving trust in context-aware systems by displaying system confidence , 2005, Mobile HCI.

[11]  Harry van der Hulst,et al.  Autosegmental Studies on Pitch Accent , 1988 .

[12]  Alfred Mertins,et al.  Automatic speech recognition and speech variability: A review , 2007, Speech Commun..

[13]  Jefferson M. Koonce,et al.  Human–automation interaction: Research and practice. , 1997 .

[14]  Mark Sanderson,et al.  Information retrieval system evaluation: effort, sensitivity, and reliability , 2005, SIGIR '05.

[15]  A. D. Dominicis,et al.  Intonation Systems: A Survey of Twenty Languages , 1999 .

[16]  Andrew Sears,et al.  Using confidence scores to improve hands-free speech based navigation in continuous dictation systems , 2004, TCHI.

[17]  Atsunori Ogawa,et al.  Joint estimation of confidence and error causes in speech recognition , 2012, Speech Commun..