The Impact of Word, Multiple Word, and Sentence Input on Virtual Keyboard Decoding Performance

Entering text on non-desktop computing devices is often done via an onscreen virtual keyboard. Input on such keyboards normally consists of a sequence of noisy tap events that specify some amount of text, most commonly a single word. But is single word-at-a-time entry the best choice? This paper compares user performance and recognition accuracy of word-at-a-time, phrase-at-a-time, and sentence-at-a-time text entry on a smartwatch keyboard. We evaluate the impact of differing amounts of input in both text copy and free composition tasks. We found providing input of an entire sentence significantly improved entry rates from 26 wpm to 32 wpm while keeping character error rates below 4%. In offline experiments with more processing power and memory, sentence input was recognized with a much lower 2.0% error rate. Our findings suggest virtual keyboards can enhance performance by encouraging users to provide more input per recognition event.

[1]  Per Ola Kristensson,et al.  Getting it right the second time: Recognition of spoken corrections , 2010, 2010 IEEE Spoken Language Technology Workshop.

[2]  Nestor Garay-Vitoria,et al.  Text prediction systems: a survey , 2006, Universal Access in the Information Society.

[3]  Shumin Zhai,et al.  Relaxing stylus typing precision by geometric pattern matching , 2005, IUI.

[4]  Yuanchun Shi,et al.  Is it too small?: Investigating the performances and preferences of users when typing on tiny QWERTY keyboards , 2017, Int. J. Hum. Comput. Stud..

[5]  Steve Young,et al.  Token passing: a simple conceptual model for connected speech recognition systems , 1989 .

[6]  Per Ola Kristensson,et al.  Discrete and Continuous Shape Writing for Text Entry and Control , 2007 .

[7]  Shumin Zhai,et al.  Quasi-qwerty soft keyboard optimization , 2010, CHI.

[8]  Shumin Zhai,et al.  Both complete and correct?: multi-objective optimization of touchscreen keyboard , 2014, CHI.

[9]  I. Scott MacKenzie,et al.  Phrase sets for evaluating text entry techniques , 2003, CHI Extended Abstracts.

[10]  Joshua Goodman,et al.  Language modeling for soft keyboards , 2002, IUI '02.

[11]  Alireza Sahami Shirazi,et al.  Text Entry on Tiny QWERTY Soft Keyboards , 2015, CHI.

[12]  Kumiko Tanaka-Ishii,et al.  Text Entry Systems: Mobility, Accessibility, Universality , 2007 .

[13]  Shumin Zhai,et al.  Optimizing Touchscreen Keyboards for Gesture Typing , 2015, CHI.

[14]  Per Ola Kristensson,et al.  Uncertain text entry on mobile devices , 2014, CHI.

[15]  Andreas Stolcke,et al.  Entropy-based Pruning of Backoff Language Models , 2000, ArXiv.

[16]  Per Ola Kristensson,et al.  Improving two-thumb text entry on touchscreen devices , 2013, CHI.

[17]  Per Ola Kristensson,et al.  Complementing text entry evaluations with a composition task , 2014, TCHI.

[18]  Per Ola Kristensson,et al.  Five Challenges for Intelligent Text Entry Methods , 2009, AI Mag..

[19]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[20]  Per Ola Kristensson,et al.  Parakeet: a continuous speech recognition system for mobile touch-screen devices , 2009, IUI.

[21]  Per Ola Kristensson,et al.  A versatile dataset for text entry evaluations based on genuine mobile emails , 2011, Mobile HCI.

[22]  Shumin Zhai,et al.  The metropolis keyboard - an exploration of quantitative techniques for virtual keyboard design , 2000, UIST '00.

[23]  Shumin Zhai,et al.  Performance Optimization of Virtual Keyboards , 2002, Hum. Comput. Interact..

[24]  Per Ola Kristensson.  Next-Generation Text Entry , 2015, Computer.

[25]  Barbara S. Chaparro,et al.  Text Input on a Smartwatch QWERTY Keyboard: Tap vs. Trace , 2017, Int. J. Hum. Comput. Interact..

[26]  Per Ola Kristensson,et al.  VelociTap: Investigating Fast Mobile Text Entry using Sentence-Based Decoding of Touchscreen Keyboard Input , 2015, CHI.

[27]  Shumin Zhai,et al.  Shorthand writing on stylus keyboard , 2003, CHI '03.

[28]  Shumin Zhai,et al.  Multilingual Touchscreen Keyboard Design and Optimization , 2012, Hum. Comput. Interact..

[29]  Shumin Zhai,et al.  The word-gesture keyboard: reimagining keyboard interaction , 2012, CACM.

[30]  Thorsten Brants,et al.  Study on interaction between entropy pruning and kneser-ney smoothing , 2010, INTERSPEECH.

[31]  Per Ola Kristensson,et al.  The inviscid text entry rate and its application as a grand goal for mobile text entry , 2014, MobileHCI '14.

[32]  Shumin Zhai,et al.  In search of effective text input interfaces for off the desktop computing , 2005, Interact. Comput..

[33]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[34]  Shumin Zhai,et al.  WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding , 2016, CHI.

[35]  Per Ola Kristensson,et al.  Performance comparisons of phrase sets and presentation styles for text entry evaluations , 2012, IUI '12.

[36]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[37]  Xiaojun Bi,et al.  Word Clarity as a Metric in Sampling Keyboard Test Sets , 2017, CHI.

[38]  Jochen Rick,et al.  Performance optimizations of virtual keyboards for stroke-based text entry on a touch-based tabletop , 2010, UIST.

[39]  Per Ola Kristensson,et al.  Automatic selection of recognition errors by respeaking the intended text , 2009, 2009 IEEE Workshop on Automatic Speech Recognition & Understanding.

[40]  I. Scott MacKenzie,et al.  Text Entry for Mobile Computing: Models and Methods,Theory and Practice , 2002, Hum. Comput. Interact..

[41]  Shumin Zhai,et al.  Movement model, hits distribution and learning in virtual keyboarding , 2002, CHI.

[42]  I. Scott MacKenzie,et al.  The design and evaluation of a high-performance soft keyboard , 1999, CHI '99.

[43]  Ali Mazalek,et al.  A Survey of Text Entry Techniques for Smartwatches , 2016, HCI.

[44]  Shumin Zhai,et al.  Interlaced QWERTY: accommodating ease of visual search and input flexibility in shape writing , 2008, CHI.