Comparing Smartphone Speech Recognition and Touchscreen Typing for Composition and Transcription

Ruan et al. found transcribing short phrases with speech recognition nearly 200% faster than typing on a smartphone. We extend this comparison to a novel composition task, using a protocol that enables a controlled comparison with transcription. Results show that both composing and transcribing with speech is faster than typing. But, the magnitude of this difference is lower with composition, and speech has a lower error rate than keyboard during composition, but not during transcription. When transcribing, speech outperformed typing in most NASA-TLX measures, but when composing, there were no significant differences between typing and speech for any measure except physical demand.

[1]  Per Ola Kristensson,et al.  Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices , 2017, CHI.

[2]  Barbara S. Chaparro,et al.  Smartphone Text Input Method Performance, Usability, and Preference With Younger and Older Adults , 2015, Hum. Factors.

[3]  I. Scott MacKenzie,et al.  Text Entry for Mobile Computing: Models and Methods,Theory and Practice , 2002, Hum. Comput. Interact..

[4]  Chong Wang,et al.  Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin , 2015, ICML.

[5]  Per Ola Kristensson,et al.  Parakeet: a continuous speech recognition system for mobile touch-screen devices , 2009, IUI.

[6]  I. Scott MacKenzie,et al.  Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric , 2003, CHI '03.

[7]  Mark D. Dunlop,et al.  Inviscid Text Entry and Beyond , 2016, CHI Extended Abstracts.

[8]  S. Rochester.  The significance of pauses in spontaneous speech , 1973, Journal of psycholinguistic research.

[9]  I. Scott MacKenzie,et al.  Measuring errors in text entry tasks: an application of the Levenshtein string distance statistic , 2001, CHI Extended Abstracts.

[10]  R. William Soukoreff,et al.  Text entry for mobile computing: models and methods , 2002 .

[11]  Allen Newell,et al.  The keystroke-level model for user performance time with interactive systems , 1980, CACM.

[12]  Paul A. Cairns,et al.  Tlk or txt? Using voice input for SMS composition , 2008, Personal and Ubiquitous Computing.

[13]  I. Scott MacKenzie,et al.  Predicting text entry speed on mobile phones , 2000, CHI.

[14]  Per Ola Kristensson,et al.  VelociWatch: Designing and Evaluating a Virtual Keyboard for the Input of Challenging Text , 2019, CHI.

[15]  Mark D. Dunlop,et al.  Measuring Inviscid Text Entry Using Image Description Tasks , 2016 .

[16]  Omer Tsimhoni,et al.  Address Entry While Driving: Speech Recognition Versus a Touch-Screen Keyboard , 2004, Hum. Factors.

[17]  Per Ola Kristensson,et al.  Complementing text entry evaluations with a composition task , 2014, TCHI.

[18]  James A. Landay,et al.  Comparing Speech and Keyboard Text Entry for Short Messages in Two Languages on Touchscreen Phones , 2016, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[19]  Per Ola Kristensson,et al.  The inviscid text entry rate and its application as a grand goal for mobile text entry , 2014, MobileHCI '14.

[20]  Per Ola Kristensson,et al.  Uncertain text entry on mobile devices , 2014, CHI.

[21]  Per Ola Kristensson,et al.  Text blaster: a multi-player touchscreen typing game , 2014, CHI Extended Abstracts.

[22]  Andrew Sears,et al.  Data Entry on the Move: An Examination of Nomadic Speech-Based Text Entry , 2004, User Interfaces for All.

[23]  Per Ola Kristensson,et al.  A versatile dataset for text entry evaluations based on genuine mobile emails , 2011, Mobile HCI.

[24]  Fiona Lyddy,et al.  An Analysis of Language in University Students' Text Messages , 2014, J. Comput. Mediat. Commun..

[25]  Virginia Z. Ogozalek,et al.  Comparison of elderly and younger users on keyboard and voice input computer-based composition tasks , 1986, CHI '86.

[26]  Daniel B. Horn,et al.  Patterns of entry and correction in large vocabulary continuous speech recognition systems , 1999, CHI '99.

[27]  Per Ola Kristensson,et al.  The Impact of Word, Multiple Word, and Sentence Input on Virtual Keyboard Decoding Performance , 2018, CHI.

[28]  Thomas Grechenig,et al.  Hyper Typer: A Serious Game for Measuring Mobile Text Entry Performance in the Wild , 2019, CHI Extended Abstracts.

[29]  I. Scott MacKenzie,et al.  Phrase sets for evaluating text entry techniques , 2003, CHI Extended Abstracts.

[30]  Christina L. James,et al.  Text input for mobile devices: comparing model prediction to actual performance , 2001, CHI.

[31]  Ben Shneiderman,et al.  The limits of speech recognition , 2000, CACM.

[32]  Brad A. Myers,et al.  An alternative to push, press, and tap-tap-tap: gesturing on an isometric joystick for mobile phone text entry , 2007, CHI.