Identifying User Corrections Automatically in Spoken Dialogue Systems

We present results of machine learning experiments designed to identify user corrections of speech recognition errors in a corpus collected from a train information spoken dialogue system. We investigate the predictive power of features automatically computable from the prosody of the turn, the speech recognition process, experimental conditions, and the dialogue history. Our best performing features reduce classification error from baselines of 25.70-28.99% to 15.72%.

[1]  Marilyn A. Walker,et al.  Automatic Detection of Poor Speech Recognition at the Dialogue Level , 1999, ACL.

[2]  Sharon L. Oviatt,et al.  Modeling hyperarticulate speech during human-computer error resolution , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[3]  Hagen Soltau,et al.  Specialized acoustic models for hyperarticulated speech , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[4]  Julia Hirschberg,et al.  Corrections in spoken dialogue systems , 2000, INTERSPEECH.

[5]  Joakim Gustafson,et al.  Repetition and its phonetic realizations : investigating a Swedish database of spontaneous computer directed speech , 1999 .

[6]  E. Russell Ritenour,et al.  Evaluating spoken dialog systems for telecommunication services , 1997, EUROSPEECH.

[7]  Shimei Pan,et al.  Empirically Evaluating an Adaptable Spoken Dialogue System , 1999, ArXiv.

[8]  William W. Cohen.  Learning Trees and Rules with Set-Valued Features , 1996, AAAI/IAAI, Vol. 1.

[9]  Julia Hirschberg,et al.  Predicting Automatic Speech Recognition Performance Using Prosodic Cues , 2000, ANLP.

[10]  Marilyn A. Walker,et al.  Learning to Predict Problematic Situations in a Spoken Dialogue System: Experiments with How May I Help You? , 2000, ANLP.

[11]  Elizabeth Shriberg,et al.  User behaviors affecting speech recognition , 1992, ICSLP.

[12]  Gina-Anne Levow,et al.  Characterizing and Recognizing Spoken Corrections in Human-Computer Dialogue , 1998, ACL.

[13]  S.J.J. Smith,et al.  Empirical Methods for Artificial Intelligence , 1995 .

[14]  Marilyn A. Walker,et al.  Towards developing general models of usability with PARADISE , 2000, Natural Language Engineering.

[15]  Julia Hirschberg,et al.  Generalizing prosodic prediction of speech recognition errors , 2000, INTERSPEECH.

[16]  Emiel Krahmer,et al.  Error spotting in human-machine interaction , 1999 .