Spoken Conversational Interaction for Language Learning

This paper describes our efforts towards utilizing multilingual spoken dialogue systems as an aid to second language acquisition. We argue that it is important for language students to have the opportunity to practice communication in a non-threatening environment, something that a computer can naturally provide. We envision a three-stage interaction focused around a specific topic of a lesson plan. The first stage would familiarize the student with the vocabulary and syntax by presenting simulated dialogues at a Web page. The second stage would involve spoken dialogue interaction with the computer, either at a workstation or on the telephone. The third stage would provide feedback to the user on the quality of the utterances they recorded during the dialogue exchange. We have thus far concentrated on Mandarin and English as the two languages, where the system can be configured reversibly to support either learning English or Mandarin. We have begun to develop dialogue interaction capabilities for a number of domains centered around the scenario of a traveler to a

[1]  James R. Glass,et al.  A probabilistic framework for feature-based speech recognition , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[2]  Victor Zue,et al.  YINHE: a Mandarin Chinese version of the GALAXY system , 1997, EUROSPEECH.

[3]  Victor Zue,et al.  GALAXY-II: a reference architecture for conversational system development , 1998, ICSLP.

[4]  Maxine Eskenazi,et al.  USING AUTOMATIC SPEECH PROCESSING FOR FOREIGN LANGUAGE PRONUNCIATION TUTORING: SOME ISSUES AND A PROTOTYPE , 1999 .

[5]  V. Melissa Holland,et al.  Preliminary Tests of Language Learning in a Speech-Interactive Graphics Microworld. , 1999 .

[6]  Victor Zue,et al.  Conversational interfaces: advances and challenges , 1997, Proceedings of the IEEE.

[7]  Stephanie Seneff,et al.  Improved tone recognition by normalizing for coarticulation and intonation effects , 2000, INTERSPEECH.

[8]  Stephanie Seneff,et al.  Robust pitch tracking for prosodic modeling in telephone speech , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[9]  Victor Zue,et al.  MUXING: a telephone-access Mandarin conversational system , 2000, INTERSPEECH.

[10]  Victor Zue,et al.  JUPlTER: a telephone-based conversational interface for weather information , 2000, IEEE Trans. Speech Audio Process..

[11]  Helmer Strik,et al.  Effective feedback on L2 pronunciation in ASR-based CALL , 2001 .

[12]  Min Tang,et al.  Voice transformations: from speech synthesis to mammalian vocalizations , 2001, INTERSPEECH.

[13]  James R. Glass,et al.  Information-theoretic criteria for unit selection synthesis , 2002, INTERSPEECH.

[14]  Joseph Polifroni,et al.  Recognition confidence scoring and its use in speech understanding systems , 2002, Comput. Speech Lang..

[15]  Chian Chuu LIESHOU : A Mandarin Conversational Task Agent for the Galaxy-II Architecture , 2003 .

[16]  Tien-Lok Jonathan Lau SLLS: An Online Conversational Spoken Language Learning System , 2003 .

[17]  Stephanie Seneff,et al.  High-quality Speech Translation for Language Learning , 2004 .

[18]  Stephanie Seneff,et al.  High-quality speech translation in the flight domain , 2006, INTERSPEECH.