Speech-enabled card games for incidental vocabulary acquisition in a foreign language

In this paper, we present a novel application for speech technology to aid students with vocabulary acquisition in a foreign language through interactive card games. We describe a generic platform for card game development and then introduce a particular prototype card game called Word War, designed for learning Mandarin Chinese. We assess the feasibility of deploying Word War via the Internet by conducting our first user study remotely and evaluating the performance of the speech recognition component. It was found that the three central concepts in our system were recognized with an error rate of 16.02%. We then turn to assessing the effects of the Word War game on vocabulary retention in a controlled environment. To this end, we performed a user study using two variants of the Word War game: a speaking mode, in which the user issues spoken commands to manipulate the game cards, and a listening mode, in which the computer gives spoken directions that the students must follow by manipulating the cards manually with the mouse. These two modes of learning were compared against a more traditional computer assisted vocabulary learning system: an on-line flash cards program. To assess long-term learning gains as a function of time-on-task, we had the students interact with each system twice over a period of three weeks. We found that all three systems were competitive in terms of the vocabulary words learned as measured by pre-tests and post-tests, with less than a 5% difference among the systems' average overall learning gains. We also conducted surveys, which indicated that the students enjoyed the speaking mode of Word War more than the other two systems.

[1]  Stephanie Seneff,et al.  Immersive second language acquisition in narrow domains: a prototype ISLAND dialogue system , 2007, SLaTE.

[2]  Patricia A. Herman,et al.  Breadth and depth of vocabulary knowledge: Implications for acquisition and instruction. , 1987 .

[3]  M. Swain.  Communicative competence : Some roles of comprehensible input and comprehensible output in its development , 1985 .

[4]  Stephen Krashen,et al.  The Input Hypothesis: Issues and Implications , 1986 .

[5]  R. Oxford,et al.  Vocabulary Learning: A Critical Analysis of Techniques , 1990 .

[6]  S. Carey.  The child as word learner , 1978 .

[7]  Stacy Marsella,et al.  Tactical Language Training System: An Interim Report , 2004, Intelligent Tutoring Systems.

[8]  Fred L. Perry,et al.  A Comparison of Three Learning Strategies for ESL Vocabulary Acquisition , 1991 .

[9]  Rod Ellis,et al.  Classroom Interaction, Comprehension, and the Acquisition of L2 Word Meanings , 1994 .

[10]  Manfred Pienemann,et al.  Constructing an Acquisition-Based Procedure for Second Language Assessment , 1988, Studies in Second Language Acquisition.

[11]  Maxine Eskenazi,et al.  USING AUTOMATIC SPEECH PROCESSING FOR FOREIGN LANGUAGE PRONUNCIATION TUTORING: SOME ISSUES AND A PROTOTYPE , 1999 .

[12]  Farzad Ehsani,et al.  Speech Technology in Computer-Assisted Language Learning: Strengths and Limitations of a New CALL Paradigm. , 1998 .

[13]  Paul Pimsleur,et al.  How to Learn a Foreign Language , 1982 .

[14]  James R. Glass.  A probabilistic framework for segment-based speech recognition , 2003, Comput. Speech Lang..

[15]  Rebecca Hincks.  Speech technologies for pronunciation feedback and evaluation , 2003, ReCALL.

[16]  John R. Anderson,et al.  Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect , 2005, Cogn. Sci..

[17]  John Nerbonne,et al.  Morphological Processing and Computer-Assisted Language Learning , 1998 .

[18]  Stephanie Seneff,et al.  Speech-enabled Card Games for Language Learners , 2008, AAAI.

[19]  R. Ellis.  Modified Oral Input and the Acquisition of Word Meanings. , 1995 .

[20]  Johann Gamper,et al.  A Review of Intelligent CALL Systems , 2002 .

[21]  Stephen Krashen,et al.  Principles and Practice in Second Language Acquisition , 1982 .

[22]  Helmer Strik,et al.  ASR-based corrective feedback on pronunciation: does it really work? , 2006, INTERSPEECH.

[23]  William E. Nagy,et al.  The Vocabulary Conundrum. , 1992 .

[24]  James R. Glass,et al.  A flexible, scalable finite-state transducer architecture for corpus-based concatenative speech synthesis , 2000, INTERSPEECH.

[25]  Amir Najmi,et al.  Subarashii: Encounters in Japanese Spoken Language Education , 1999 .

[26]  Rod Ellis,et al.  Modified Input and the Acquisition of Word meaning by Children and Adults , 1999 .

[27]  N. Ellis,et al.  Implicit and explicit learning of languages , 1997 .

[28]  Wolfgang Menzel,et al.  Interactive pronunciation training , 2001, ReCALL.

[29]  Meng Zhang,et al.  The WISTON Text to Speech System for Blizzard 2008 , 2008 .

[30]  Michael M. Gruneberg,et al.  Individual differences and attitudes to the keyword method of foreign language learning , 1991 .

[31]  Maxine Eskénazi,et al.  The NativeaccentTM pronunciation tutor: measuring success in the real world , 2007, SLaTE.

[32]  Edwin T. Cornelius.  How to learn a foreign language , 1955 .

[33]  Helmer Strik,et al.  ASR corrective feedback on pronunciation: Does it really work? , 2006 .

[34]  V. Melissa Holland,et al.  Preliminary Tests of Language Learning in a Speech-Interactive Graphics Microworld. , 1999 .

[35]  Luis von Ahn.  Games with a Purpose , 2006, Computer.

[36]  Roger Cooley,et al.  Vocabulary acquisition software: user preferences and tutorial guidance , 2001 .

[37]  Ian McGraw,et al.  The WAMI toolkit for developing, deploying, and evaluating web-accessible multimodal interfaces , 2008, ICMI '08.

[38]  Susan M. Gass,et al.  Input in second language acquisition , 1985 .

[39]  Michael H. Long.  INPUT, INTERACTION, AND SECOND‐LANGUAGE ACQUISITION , 1981 .

[40]  Rod Ellis,et al.  THE ROLES OF MODIFIED INPUT AND OUTPUT IN THE INCIDENTAL ACQUISITION OF WORD MEANINGS , 1999, Studies in Second Language Acquisition.

[41]  Robert C. Duncan,et al.  Virtual Conversations: An Interface to Knowledge , 2003, IEEE Computer Graphics and Applications.

[42]  Stephanie Seneff,et al.  Towards Automatic Tone Correction in Non-native Mandarin , 2006, ISCSLP.

[43]  I. S. P. Nation,et al.  Learning Vocabulary in Another Language: Appendixes , 2001 .