New challenges in usability evaluation - beyond task-oriented spoken dialogue systems

There is a fairly good baseline for usability evaluation of taskoriented unimodal spoken dialogue systems (SDSs) but much is still unknown regarding the usability of multimodal and non-task-oriented SDSs. This paper reviews and discusses approaches to usability evaluation of these kinds of SDSs.

[1]  Janienke Sturm,et al.  Effects of prolonged use on the usability of a multimodal form-filling interface , 2004 .

[2]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[3]  Niels Ole Bernsen,et al.  Usability Issues in Spoken Language Dialogue Systems , 2001 .

[4]  Julia Hirschberg,et al.  Labeling Corrections and Aware Sites in Spoken Dialogue Systems , 2001, SIGDIAL Workshop.

[5]  Niels Ole Bernsen,et al.  First prototype of conversational H.C. Andersen , 2004, AVI.

[6]  Björn Granström,et al.  Effective Interaction with Talking Animated Agents an Dialogue Systems , 2005 .

[7]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[8]  Anthony Jameson,et al.  User Multitasking with Mobile Multimodal Systems , 2005 .

[9]  Janienke Sturm,et al.  Comparing the usability of a user driven and a mixed initiative multimodal dialogue system for train timetable information , 2003, INTERSPEECH.

[10]  Wolfgang Minker,et al.  The SENECA spoken language dialogue system , 2004, Speech Commun..

[11]  J. Cassell,et al.  Social Dialongue with Embodied Conversational Agents , 2005 .

[12]  K. Fischer,et al.  DESPERATELY SEEKING EMOTIONS OR: ACTORS, WIZARDS, AND HUMAN BEINGS , 2000 .

[13]  J. Cassell,et al.  SOCIAL DIALOGUE WITH EMBODIED CONVERSATIONAL AGENTS , 2005 .

[14]  Niels Ole Bernsen,et al.  Multimodality in Language and Speech Systems — From Theory to Design Support Tool , 2002 .

[15]  Niels Ole Bernsen,et al.  Evaluation and usability of multimodal spoken language dialogue systems , 2004, Speech Commun..

[16]  Ronald A. Cole,et al.  Perceptive animated interfaces: first steps toward a new paradigm for human-computer interaction , 2003, Proc. IEEE.

[17]  Niels Ole Bernsen.  User Modelling in the Car , 2003, User Modeling.

[18]  Joakim Gustafson,et al.  The august spoken dialogue system , 1999, EUROSPEECH.

[19]  Dominic W. Massaro,et al.  The Psychology and Technology of Talking Heads: Applications in Language Learning , 2005 .

[20]  Philip R. Cohen,et al.  The Efficiency of Multimodal Interaction for a Map-based Task , 1999, ANLP.

[21]  Nicole Beringer,et al.  PROMISE - A Procedure for Multimodal Interactive System Evaluation , 2002 .

[22]  Marilyn A. Walker,et al.  PARADISE: A Framework for Evaluating Spoken Dialogue Agents , 1997, ACL.

[23]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[24]  Niels Ole Bernsen,et al.  Evaluating Conversation with Hans Christian Andersen , 2004, LREC.

[25]  Lou Boves.  Usability of a Speech Centric Multimodal Directory Assistance Service , 2001 .

[26]  Dirk Heylen,et al.  CONTROLLING THE GAZE OF CONVERSATIONAL AGENTS , 2005 .

[27]  Laila Dybkjær,et al.  Affective Dialogue Systems , 2004, Lecture Notes in Computer Science.

[28]  Susanne van Mulken,et al.  The impact of animated interface agents: a review of empirical research , 2000, Int. J. Hum. Comput. Stud..

[29]  Laila Dybkjær,et al.  Spoken Multimodal Human-Computer Dialogue in Mobile Environments , 2005 .

[30]  Wolfgang Minker,et al.  Flexible multimodal human-machine interaction in mobile environments , 2002, INTERSPEECH.