Human Factors and Design Issues in Multimodal (Speech/Gesture) Interface

Multimodal interfaces are the emerging technology that offers expressive, transparent, efficient, robust, and mobile human-computer interaction. In this paper, we described the speech/gesture based multimodal interface systematically from the human factors point of view. To design more practical and efficient multimodal interface, human factors issues such as user modeling, usability studies, speech and gesture interaction/integration, and redundancy are discussed. This paper can be helpful to the researchers in the field of study that enhances the performance of multimodal interaction.

[1]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[2]  James H. Bradford,et al.  The human factors of speech-based interfaces: a research agenda , 1995, SGCH.

[3]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[4]  Mohammed Yeasin,et al.  Speech-gesture driven multimodal interfaces for crisis management , 2003, Proc. IEEE.

[5]  Elizabeth D. Mynatt.  Transforming graphical interfaces into auditory interfaces , 1995, CHI 95 Conference Companion.

[6]  Ronald R. Mourant,et al.  Human Factors Issues in Virtual Environments: A Review of the Literature , 1998, Presence.

[7]  K. Stanney,et al.  Human factors evaluation of virtual environments , 1996, Southcon/96 Conference Record.

[8]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[9]  H. Shahnavaz,et al.  A Usability Evaluation of Text and Speech Redundant Help Messages on a Reader Interface , 1993, HCI.

[10]  Robert I. Damper,et al.  Speech as an interface medium: how can it best be used? , 1993 .

[11]  Chris Baber,et al.  Interactive speech technology: human factors issues in the application of speech input/output to computers , 1993 .

[12]  Alexander H. Waibel,et al.  Multimodal interfaces , 1996, Artificial Intelligence Review.

[13]  K. Gegenfurtner,et al.  Design Issues in Gaze Guidance Under review with ACM Transactions on Computer Human Interaction , 2009 .

[14]  Sherry Perdue Casali,et al.  Effects of Recognition Accuracy and Vocabulary Size of a Speech Recognition System on Task Performance and User Acceptance , 1990 .

[15]  Lambert Schomaker,et al.  A taxonomy of Multimodal Interaction in the Human Information Processing System , 1995 .

[16]  Victoria Hoban,et al.  The Reflective Practitioner , 2013 .

[17]  D MynattElizabeth,et al.  Charting past, present, and future research in ubiquitous computing , 2000 .

[18]  D. Schoen.  The Reflective Practitioner , 1983 .

[19]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[20]  J. Jacko,et al.  The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications , 2002 .