论文信息 - A Task-Based Evaluation of the TRAINS-95 Dialogue System

A Task-Based Evaluation of the TRAINS-95 Dialogue System

This paper describes a task-based evaluation methodology appropriate for dialogue systems such as the TRAINS-95 system, where a human and a computer interact and collaborate to solve a given problem. In task-based evaluations, techniques are measured in terms of their effect on task performance measures such as how long it takes to develop a solution using the system, and the quality of the final plan produced. We report recent experiment results which explore the effect of word recognition accuracy on task performance.

James F. Allen | Teresa Sikorski | Teresa Sikorski

[1] Günther Görz,et al. Towards understanding spontaneous speech: word accuracy vs. concept accuracy , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[2] Eric K. Ringger,et al. Error correction via a post-processor for continuous speech recognition , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[3] Elizabeth Shriberg,et al. Human-Machine Problem Solving Using Spoken Language Systems (SLS): Factors Affecting Performance and User Satisfaction , 1992, HLT.

[4] Philip R. Cohen,et al. The contributing influence of speech and interaction on human discourse patterns , 1991 .

[5] Victor Zue,et al. Experiments in Evaluating Interactive Spoken Language Systems , 1992, HLT.

[6] D. Richard Hipp,et al. Spoken Natural Language Dialog Systems: A Practical Approach , 1994 .

[7] Sharon M. Walter. Neal-Montgomery NLP System Evaluation Methodology , 1992, HLT.

[8] Mei-Yuh Hwang,et al. The SPHINX-II speech recognition system: an overview , 1993, Comput. Speech Lang..

[9] P R Cohen,et al. The role of voice input for human-machine communication. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[10] Alexander I. Rudnicky,et al. Multi-Site Data Collection and Evaluation in Spoken Language Understanding , 1993, HLT.