Simple Construction of Mixed-Language Texts for Vocabulary Learning

We present a machine foreign-language teacher that takes documents written in a student’s native language and detects situations where it can replace words with their foreign glosses such that new foreign vocabulary can be learned simply through reading the resulting mixed-language text. We show that it is possible to design such a machine teacher without any supervised data from (human) students. We accomplish this by modifying a cloze language model to incrementally learn new vocabulary items, and use this language model as a proxy for the word guessing and learning ability of real students. Our machine foreign-language teacher decides which subset of words to replace by consulting this language model. We evaluate three variants of our student proxy language models through a study on Amazon Mechanical Turk (MTurk). We find that MTurk “students” were able to guess the meanings of foreign words introduced by the machine teacher with high accuracy for both function words as well as content words in two out of the three models. In addition, we show that students are able to retain their knowledge about the foreign words after they finish reading the document.

[1]  Richard Socher,et al.  Pointer Sentinel Mixture Models , 2016, ICLR.

[2]  Luis von Ahn.  Duolingo: learn a language for free while helping to translate the web , 2013, IUI '13.

[3]  Björn W. Schuller,et al.  Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis , 2017, EACL.

[4]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  Hod Lipson,et al.  Generating Code-switched Text for Lexical Learning , 2014, ACL.

[7]  Alexander M. Rush,et al.  Character-Aware Neural Language Models , 2015, AAAI.

[8]  Philipp Koehn,et al.  User Modeling in Language Learning with Macaronic Texts , 2016, ACL.

[9]  Sampo Pyysalo,et al.  Universal Dependencies v1: A Multilingual Treebank Collection , 2016, LREC.

[10]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[11]  J. Bruner,et al.  The role of tutoring in problem solving. , 1976, Journal of child psychology and psychiatry, and allied disciplines.

[12]  S. Krashen.  We Acquire Vocabulary and Spelling by Reading: Additional Evidence for the Input Hypothesis , 1989 .

[13]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[16]  Jürgen Schmidhuber,et al.  Highway Networks , 2015, ArXiv.

[17]  Zili Huang,et al.  Recover Missing Sensor Data with Iterative Imputing Network , 2017, AAAI Workshops.