A First-Personness Approach to Co-operative Multimodal Interaction

Using natural language in addition to graphical user interfaces is often used as an argument for a better interaction. However, just adding spoken language might not lead to a better interaction. In this article we will look deeper into how the spoken language should be used in a cooperative multimodal interface. Based on empirical investigations, we have noticed that for multimodal information systems efficiency is especially important. Our results indicate that efficiency can be divided into functional and linguistic efficiency. Functional efficiency has a tight relation to solving the task fast. Linguistic efficiency concerns how to make the contributions meaningful and appropriate in the context. For linguistic efficiency user's perception of first-personness [1] is important, as well as giving users support for understanding the interface, and to adapt the responses to the user. In this article focus is on linguistic efficiency for a multimodal timetable information system.

[1]  H. Grice.  Logic and conversation , 1975 .

[2]  Norman M. Fraser,et al.  Dialogue Management for Telephone Information Systems , 1992, ANLP.

[3]  David Williams,et al.  “Just speak naturally”: designing for naturalness in automated spoken dialogues , 1998, CHI Conference Summary.

[4]  Arne Jönsson,et al.  AN ARCHITECTURE FOR MULTI-MODAL NATURAL DIALOGUE SYSTEMS , 2000 .

[5]  Jennifer Chu-Carroll.  Evaluating Automatic Dialogue Strategy Adaptation for a Spoken Dialogue System , 2000, ANLP.

[6]  Harry Bunt,et al.  Multimodal Cooperation with the DENK System , 1995, Multimodal Human-Computer Communication.

[7]  S. Joy Mountford,et al.  The Art of Human-Computer Interface Design , 1990 .

[8]  Raj Reddy,et al.  Steps Toward Graceful Interaction in Spoken and Written Man-Machine Communication , 1983, Int. J. Man Mach. Stud..

[9]  Arne Jönsson,et al.  Dialogue and Domain Knowledge Management in Dialogue Systems , 2000, SIGDIAL Workshop.

[10]  Herbert H. Clark,et al.  Contributing to Discourse , 1989, Cogn. Sci..

[11]  Donald A. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[12]  Mark T. Maybury,et al.  Toward Cooperative Multimedia Interaction , 1995, Multimodal Human-Computer Communication.

[13]  Annika Flycht-Eriksson,et al.  A domain knowledge manager for dialogue systems , 2000 .

[14]  Candace L. Sidner,et al.  Lessons Learned in Building Spoken Language Collaborative Interface Agents , 2000 .

[15]  Arne Jönsson,et al.  Talking to a Computer Is Not like Talking to Your Best Friend , 1988, SCAI.

[16]  Manuel A. Pérez-Quiñones,et al.  A collaborative model of feedback in human-computer interaction , 1996, CHI.

[17]  Brenda K. Laurel,et al.  Interface as Mimesis , 1986 .

[18]  Laurel Fais,et al.  Speakers' Responses to Requests for Repetition in a Multimedia Language Processing Environment , 1995, Multimodal Human-Computer Communication.

[19]  C. Cheepen,et al.  Designing for Naturalness in Automated Dialogues , 1999 .

[20]  Susan Brennan,et al.  Interaction and feedback in a spoken language system: a theoretical framework , 1995, Knowl. Based Syst..