A recursive dialogue game framework with optimal Policy offering personalized computer-assisted language learning

This paper introduces a new recursive dialogue game framework for personalized computer-assisted language learning. A series of sub-dialogue trees are cascaded into a loop as the script for the game. At each dialogue turn there are a number of training sentences to be selected. The dialogue policy is optimized to offer the most appropriate training sentence for an individual learner at each dialogue turn considering the learning status, such that the learner can have the scores for all pronunciation units exceeding a pre-defined threshold in minimum number of turns. The policy is modeled as a Markov Decision Process (MDP) with high dimensional continuous state space. Experiments demonstrate promising results for the approach.

[1]  Apostolos Burnetas,et al.  Optimal Adaptive Policies for Markov Decision Processes , 1997, Math. Oper. Res..

[2]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[3]  Matthieu Geist,et al.  Off-policy learning in large-scale POMDP-based dialogue systems , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Donna Christian.  Profiles in Two-Way Immersion Education. Language in Education: Theory and Practice 89. , 1997 .

[6]  S. Merriam.  Qualitative Research and Case Study Applications in Education: Revised and Expanded from Case Study Research in Education , 1998 .

[7]  C. Wieman,et al.  Why peer discussion improves student performance on in-class concept questions , 2009 .

[8]  Zucchini,et al.  An Introduction to Model Selection. , 2000, Journal of mathematical psychology.

[9]  Yushi Xu,et al.  Language technologies in speech-enabled second language learning games: from reading to dialogue , 2012 .

[10]  W. Lewis Johnson,et al.  Serious Use of a Serious Game for Language Learning , 2007, AIED.

[11]  Lin-Shan Lee,et al.  A dialogue game framework with personalized training using reinforcement learning for computer-assisted language learning , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Stephanie Seneff,et al.  A generic framework for building dialogue games for language learning: application in the flight domain , 2011, SLaTE.

[13]  J. Schatztnann,et al.  Effects of the user model on simulation-based learning of dialogue strategies , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[14]  Gregory A Petsko.  The Rosetta Stone , 2001, Genome Biology.

[15]  Maxine Eskénazi,et al.  Incremental Sparse Bayesian Method for Online Dialog Strategy Learning , 2012, IEEE Journal of Selected Topics in Signal Processing.

[16]  Sean P. Meyn,et al.  An analysis of reinforcement learning with function approximation , 2008, ICML '08.

[17]  O. Pietquin,et al.  Optimizing Spoken Dialogue Management from Data Corpora with Fitted Value Iteration , 2010 .

[18]  Matthieu Geist,et al.  Optimizing spoken dialogue management with fitted value iteration , 2010, INTERSPEECH.

[19]  Helmer Strik,et al.  Practice and feedback in L2 speaking: an evaluation of the DISCO CALL system , 2012, INTERSPEECH.

[20]  Hua Ai,et al.  User Simulation as Testing for Spoken Dialog Systems , 2008, SIGDIAL Workshop.

[21]  J. Dewey.  Experience and Education , 1938 .

[22]  Eric Clearinghouse on Languages and Linguistics.  Profiles in Two-Way Immersion Education , 1997 .

[23]  Antoine Raux,et al.  Using Task-Oriented Spoken Dialogue Systems for Language Learning: Potential, Practical Applications and Challenges , 2004 .

[24]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[25]  Shie Mannor,et al.  Regularized Fitted Q-Iteration for planning in continuous-space Markovian decision problems , 2009, 2009 American Control Conference.

[26]  Frank K. Soong,et al.  The Use of DBN-HMMs for Mispronunciation Detection and Diagnosis in L2 English to Support Computer-Aided Pronunciation Training , 2012, INTERSPEECH.

[27]  Csaba Szepesvári,et al.  Fitted Q-iteration in continuous action-space MDPs , 2007, NIPS.

[28]  Jason D. Williams,et al.  Demonstration of AT&T “Let's Go”: A production-grade statistical spoken dialog system , 2010, 2010 IEEE Spoken Language Technology Workshop.

[29]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[30]  Shie Mannor,et al.  Bayes Meets Bellman: The Gaussian Process Approach to Temporal Difference Learning , 2003, ICML.

[31]  Maxine Eskénazi,et al.  An overview of spoken language technology for education , 2009, Speech Commun..

[32]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[33]  Keikichi Hirose,et al.  Automatic Chinese pronunciation error detection using SVM trained with structural features , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[34]  F. Downton,et al.  Introduction to Mathematical Statistics , 1959 .

[35]  Kei Hirose,et al.  Bayesian Information Criterion and Selection of the Number of Factors in Factor Analysis Models , 2021 .