Human evaluation of the LOGOS' spoken dialogue system

In this work, the evaluation of the LOGOS' spoken dialogue system is presented. The system offers user-friendly access to information, entertainment devices and white good appliances. Short description of the LOGOS system's architecture is given. The user interface of the system is based on remote control device, PC keyboard and spoken language. In this paper we focus on the speech interface and the spoken dialogue management. A group of 15 members, aged 23 to 35, were used, in order to evaluate the usability and the degree of acceptance of the LOGOS' spoken dialogue system from real users. The users were given taskcards, which constructed the scenarios implemented for the home devices' control and the SMS messaging services.

[1]  Laila Dybkjær,et al.  Usability Evaluation in Spoken Language Dialogue Systems , 2001, ACL 2001.

[2]  Li Deng,et al.  Mipad: a next generation PDA prototype , 2000, INTERSPEECH.

[3]  Marilyn A. Walker,et al.  Design and evaluation of spoken dialog systems , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[4]  Jennifer Chu-Carroll,et al.  MIMIC: An Adaptive Mixed Initiative Spoken Dialogue System for Information Queries , 2000, ANLP.

[5]  Karsten Konrad,et al.  Model Generation for Natural Language Interpretation and Analysis , 2004, Lecture Notes in Computer Science.

[6]  Panagiotis Zervas,et al.  Segmental Duration Modeling for Greek Speech Synthesis , 2007, 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007).

[7]  Markku Turunen,et al.  Evaluation of a spoken dialogue system with usability tests and long-term pilot studies: similarities and differences , 2006, INTERSPEECH.

[8]  liza. hassel,et al.  Evaluation of a Dialogue System in an Automotive Environment , 2005, SIGDIAL Workshop.

[9]  Victor W. Zue.  Automatic speech recognition and understanding , 1990 .

[10]  Victor Zue,et al.  Data collection and performance evaluation of spoken dialogue systems: the MIT experience , 2000, INTERSPEECH.

[11]  Kuansan Wang.  A plan-based dialog system with probabilistic inferences , 2000, INTERSPEECH.

[12]  Gerhard Sagerer,et al.  Evaluation of spoken language understanding and dialogue systems , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[13]  Matthias Denecke.  Rapid Prototyping for Spoken Dialogue Systems , 2002, COLING.