论文信息 - Validation of a Dialog System for Language Learners

Validation of a Dialog System for Language Learners

In this paper we present experiments related to the validation of spoken language understanding capabilities in a language and culture training system. In this application, word-level recognition rates are insufficient to characterize how well the system serves its users. We present the results of an annotation exercise that distinguishes instances of non-recognition due to learner error from instances due to poor system coverage. These statistics give a more accurate and interesting description of system performance, showing how the system could be improved without sacrificing the instructional value of rejecting learner utterances when they are poorly formed.

W. Lewis Johnson | Alicia Sagae | Stephen Bodnar

[1] James C. Lester,et al. Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[2] M. J. Hunt. Figures of merit for assessing connected-word recognisers , 1990, Speech Commun..

[3] W. Johnson,et al. Developing Serious Games for Learning Language-in-Culture , 2010 .

[4] W. Lewis Johnson,et al. Culture-driven response strategies for virtual human behavior in training systems , 2009, SLaTE.

[5] Graeme Hirst,et al. The Repair of Speech Act Misunderstandings by Abductive Inference , 1995, CL.

[6] W. Lewis Johnson,et al. Tactical Language and Culture Training Systems: Using AI to Teach Foreign Languages and Cultures , 2009, AI Mag..

[7] Ning Wang,et al. The Politeness Effect in an Intelligent Foreign Language Tutoring System , 2008, Intelligent Tutoring Systems.

[8] Marilyn A. Walker,et al. Evaluating spoken dialogue agents with PARADISE: Two case studies , 1998, Comput. Speech Lang..

[9] David Suendermann-Oeft,et al. A Handsome Set of Metrics to Measure Utterance Classification Performance in Spoken Dialog Systems , 2009, SIGDIAL Conference.