Improving support of conversations by enhancing mobile computer input

Mobile computing is becoming one of the most widely adopted technologies. There are 1.3 billion mobile phone subscribers worldwide, and the current generation of phones offers substantial computing ability. Furthermore, mobile devices are increasingly becoming integrated into everyday life. With the huge popularity in mobile computing, it is critical that we examine the human-computer interaction issues for these devices and explicitly explore supporting everyday activities. In particular, one very common and important activity of daily life I am interested in supporting is conversation. Depending on job type, office works can spend up to 85% of their time in interpersonal communication. In this work, I present two methods that improve a user's ability to enter information into a mobile computer in conversational situations. First I examine the Twiddler, a keyboard that has been adopted by the wearable computing community. The Twiddler is a mobile one-handed chording keyboard with a keypad similar to a mobile phone. The second input method is dual-purpose speech, a technique designed to leverage a user's conversational speech. A dual-purpose speech interaction is one where speech serves two roles; it is socially appropriate and meaningful in the context of a human-to-human conversation and provides useful input to a computer. A dual-purpose speech application listens to one side of a conversation and provides beneficial services to the user. Together these input methods provide a user the ability to enter information while engaged in conversation in a mobile setting.

[1]  I. Scott MacKenzie,et al.  Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric , 2003, CHI '03.

[2]  Uwe Küssner,et al.  Tracking and Understanding Temporal Descriptions in Dialogue , 1998 .

[3]  Adelino Santos,et al.  The Big Bang , 1995 .

[4]  Karen Holtzblatt,et al.  Contextual design , 1997, INTR.

[5]  George R. Doddington,et al.  The ATIS Spoken Language Systems Pilot Corpus , 1990, HLT.

[6]  Thad Starner,et al.  Use of mobile appointment scheduling devices , 2004, CHI EA '04.

[7]  I. Scott MacKenzie,et al.  The design and evaluation of a high-performance soft keyboard , 1999, CHI '99.

[8]  G. Leech,et al.  Word Frequencies in Written and Spoken English: based on the British National Corpus , 2001 .

[9]  I. Scott MacKenzie,et al.  Phrase sets for evaluating text entry techniques , 2003, CHI Extended Abstracts.

[10]  John R. Searle,et al.  Speech Acts: An Essay in the Philosophy of Language , 1970 .

[11]  Barry Arons,et al.  VoiceNotes: a speech interface for a hand-held voice notetaker , 1993, INTERCHI.

[12]  Thierry Declerck,et al.  Natural Language Dialogue Service for Appointment Scheduling Agents , 1997, ANLP.

[13]  Kent Lyons,et al.  Mobile capture for wearable computer usability testing , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[14]  Raymond R. Panko,et al.  Managerial Communication Patterns , 1992 .

[15]  Aaron E. Rosenberg,et al.  SCANMail: a voicemail interface that makes speech browsable, readable and searchable , 2002, CHI.

[16]  P. Hancock,et al.  Human Mental Workload , 1988 .

[17]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[18]  Christina L. James,et al.  Text input for mobile devices: comparing model prediction to actual performance , 2001, CHI.

[19]  Daniel J. Wigdor,et al.  TiltText: using tilt for text input to mobile phones , 2003, UIST '03.

[20]  David Frohlich.  Requirements for interpersonal information management , 1995 .

[21]  Alison Kidd,et al.  The marks are on the knowledge worker , 1994, CHI '94.

[22]  I. Scott MacKenzie,et al.  KSPC (Keystrokes per Character) as a Characteristic of Text Entry Techniques , 2002, Mobile HCI.

[23]  I. Scott MacKenzie,et al.  LetterWise: prefix-based disambiguation for mobile text input , 2001, UIST '01.

[24]  Pattie Maes,et al.  Just-in-time information retrieval , 2000 .

[25]  J. C. Byers,et al.  Comparison of Four Subjective Workload Rating Scales , 1992 .

[26]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[27]  V. Rich.  Personal communication , 1989, Nature.

[28]  Miika Silfverberg.  Using Mobile Keypads with Limited Visual Feedback: Implications to Handheld and Wearable Devices , 2003, Mobile HCI.

[29]  Alex Pentland,et al.  Wearable computing and contextual awareness , 1999 .

[30]  John Makhoul,et al.  Comparative experiments on large vocabulary speech recognition , 1993 .

[31]  C. Wickens,et al.  Dissociation of Performance and Subjective Measures of Workload , 1988 .

[32]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[33]  Eduardo Salas,et al.  Making decisions under stress: Implications for individual and team training. , 1998 .

[34]  Bill N. Schilit,et al.  Dynomite: a dynamically organized ink and audio notebook , 1998 .

[35]  I. Scott MacKenzie,et al.  Predicting text entry speed on mobile phones , 2000, CHI.

[36]  Gregory D. Abowd,et al.  Practices for capturing short important thoughts , 2003, CHI Extended Abstracts.

[37]  Ben Shneiderman,et al.  The limits of speech recognition , 2000, CACM.

[38]  P R Cohen,et al.  The role of voice input for human-machine communication. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[39]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[40]  Paul P. Maglio,et al.  Supporting notable information in office work , 2003, CHI Extended Abstracts.

[41]  Gina-Anne Levow,et al.  Designing SpeechActs: issues in speech user interfaces , 1995, CHI '95.

[42]  Kent Lyons,et al.  Twiddler typing: one-handed chording text entry for mobile phones , 2004, CHI.

[43]  John D. Gould,et al.  Composing letters with a simulated listening typewriter , 1982, CHI '82.

[44]  Kenton O'Hara,et al.  Dealing with mobility: understanding access anytime, anywhere , 2001, TCHI.

[45]  Kent Lyons.  Everyday Wearable Computer Use: A Case Study of an Expert User , 2003, Mobile HCI.

[46]  Kent Lyons,et al.  An empirical study of typing rates on mini-QWERTY keyboards , 2005, CHI Extended Abstracts.

[47]  Andy Cockburn,et al.  An Evaluation of Mobile Phone Text Input Methods , 2002, AUIC.

[48]  Catalina Danis,et al.  Storywriter: a speech oriented editor , 1994, CHI '94.

[49]  I. Scott MacKenzie,et al.  One-Handed Touch Typing on a QWERTY keyboard , 1996, Hum. Comput. Interact..

[50]  Daniel J. Wigdor,et al.  A comparison of consecutive and concurrent input text entry techniques for mobile phones , 2004, CHI.

[51]  Wayne G. Lutters,et al.  Understanding the micronote lifecycle: improving mobile support for informal note taking , 2004, CHI.

[52]  Chris Schmandt.  Voice communication with computers , 1993 .

[53]  Mei-Yuh Hwang,et al.  The SPHINX-II speech recognition system: an overview , 1993, Comput. Speech Lang..

[54]  Gregory D. Abowd,et al.  The Personal Audio Loop: Designing a Ubiquitous Audio-Based Memory Aid , 2004, Mobile HCI.

[55]  Jacqueline Brodie,et al.  Designing to support communication on the move , 2003, CHI Extended Abstracts.

[56]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[57]  Alexander I. Rudnicky.  Mode preference in a simple data-retrieval task , 1993, HLT.

[58]  Richard Catrambone,et al.  Learning a word processing system with training wheels and guided exploration , 1987, CHI '87.

[59]  Lisa Stifelman,et al.  Augmenting real-world objects: a paper-based audio notebook , 1996, CHI Conference Companion.

[60]  Owen Daly-Jones,et al.  Informal workplace communication: what is it like and how might we support it? , 1994, CHI '94.