INTERFACE FEATURES AFFECTING DEIXIS PRODUCTION: A SIMULATION STUDY

This paper examines gestural deixis production in a multimodal (natural language and pointing) interaction. Users (N=30) were required to fill in 4 forms by interacting with a simulated system. Labelling of Entry Fields (LEF, Complete-Partial; between-subject) and Feedback (Present-Absent within-subject) were manipulated in a 2*2 design. Results indicate that deixis occurred more frequently in the Partial-LEF condition. Specific input device constraints influenced intermodal integration: pointing was performed after the end of writing, although feedback presence increased modality shifts. Simulation offered a valuable tool for the development of multimodal interaction models.

[1]  Wolfgang Wahlster,et al.  Combining Deictic Gestures and Natural Language for Referent Identification , 1986, COLING.

[2]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[3]  木村 和夫.  Pragmatics , 1997, Language Teaching.

[4]  Arne Jönsson,et al.  Talking to a Computer Is Not like Talking to Your Best Friend , 1988, SCAI.

[5]  H. H. Clark,et al.  Common ground at the understanding of demonstrative reference , 1983 .

[6]  Dagmar Schmauks,et al.  Integration of communicative hand movements into human-computer-interaction , 1991, Comput. Humanit..

[7]  Mark T. Maybury,et al.  Intelligent multimedia interfaces , 1994, CHI Conference Companion.

[8]  Raymonde Guindon,et al.  Users Request Help From Advisory Systems With Simple and Restricted Language: Effects of Real-Time Constraints and Limited Shared Context , 1991, Hum. Comput. Interact..

[9]  Oliviero Stock,et al.  ALFRESCO: Enjoying the Combination of Natural Language Processing and Hypermedia for Information Exploration , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[10]  Sharon L. Oviatt,et al.  A rapid semi-automatic simulation technique for investigating interactive speech and handwriting , 1992, ICSLP.

[11]  Sharon L. Oviatt,et al.  Multimodal interfaces for dynamic interactive maps , 1996, CHI.

[12]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[13]  Sharon L. Oviatt,et al.  Integration themes in multimodal human-computer interaction , 1994, ICSLP.

[14]  Nigel Gilbert,et al.  Simulating speech systems , 1991 .

[15]  Willem J. M. Levelt,et al.  Pointing and voicing in deictic expressions , 1985 .

[16]  Sharon L. Oviatt,et al.  Toward interface design for human language technology: Modality and structure as determinants of linguistic complexity , 1994, Speech Communication.

[17]  Norbert Reithinger,et al.  Generating Multimodal Output- Conditions, Advantages and Problems , 1988, COLING.

[18]  Mark A. McDaniel,et al.  Mental models, pictures, and text: Integration of spatial and verbal information , 1992, Memory & cognition.

[19]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .