Multimodal dialogue systems for interactive TV applications

Many studies have shown the advantages of building multimodal systems, but not in the interactive TV application context. This paper reports on a qualitative study of a multimodal program guide for interactive TV. The system was designed by adding speech interaction to an existing TV program guide. Results indicate that spoken natural language input combined with visual output is preferable for TV applications. Furthermore, user feedback requires a clear distinction between the dialogue system's domain result and system status in the visual output. Consequently, we propose an interaction model that consists of three entities: user, domain results, and system feedback.

[1]  Peter Thanisch,et al.  Natural language interfaces to databases – an introduction , 1995, Natural Language Engineering.

[2]  P R Cohen,et al.  The role of voice input for human-machine communication. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Karen Renaud,et al.  Feedback in human-computer interaction - characteristics and recommendations , 2000, South Afr. Comput. J..

[4]  Virginia Teller.  Review of Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition by Daniel Jurafsky and James H. Martin. Prentice Hall 2000. , 2000 .

[5]  B. Schneirdeman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[6]  E Windisch,et al.  [South Africa]. , 1976, Osterreichische Krankenpflegezeitschrift.

[7]  Arne Jönsson,et al.  Dialogue and Domain Knowledge Management in Dialogue Systems , 2000, SIGDIAL Workshop.

[8]  Jonas Lundberg,et al.  Speech enhanced remote control for media terminal , 2001, INTERSPEECH.

[9]  Philip R. Cohen,et al.  The role of voice in human-machine communication , 1994 .

[10]  Arne Jönsson,et al.  Iterative Development of an Information- Providing Dialogue System , 2002 .

[11]  Philip R. Cohen.  The role of natural language in a multimodal interface , 1992, UIST '92.

[12]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[13]  David M. Lane,et al.  Impact of a restricted natural language interface on ease of learning and productivity , 1989, CACM.

[14]  Susan Brennan,et al.  Interaction and feedback in a spoken language system: a theoretical framework , 1995, Knowl. Based Syst..

[15]  Marilyn A. Walker,et al.  Mixed Initiative in Dialogue: An Investigation into Discourse Segmentation , 1990, ACL.

[16]  Alexander I. Rudnicky,et al.  Universal speech interfaces , 2001, INTR.

[17]  Jay G. Wilpon,et al.  Voice communication between humans and machines , 1994 .

[18]  H. H. Clark,et al.  Collaborating on contributions to conversations , 1987 .

[19]  Harry Bunt,et al.  Multimodal Cooperation with the DENK System , 1995, Multimodal Human-Computer Communication.

[20]  Gina-Anne Levow,et al.  Designing SpeechActs: issues in speech user interfaces , 1995, CHI '95.

[21]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[22]  Ben Shneiderman,et al.  Designing the user interface (2nd ed.): strategies for effective human-computer interaction , 1992 .

[23]  Philip R. Cohen,et al.  Synergistic use of direct manipulation and natural language , 1989, CHI '89.

[24]  James F. Allen,et al.  An architecture for more realistic conversational systems , 2001, IUI '01.