HotStrokes: Word-Gesture Shortcuts on a Trackpad

Expert interaction techniques like hotkeys are efficient, but poorly adopted because they are hard to learn. HotStrokes removes the need for learning arbitrary mappings of commands to hotkeys. A user enters a HotStroke by holding a modifier key, then gesture typing a command name on a laptop trackpad as if on an imaginary virtual keyboard. The gestures are recognized using an adaptation of the SHARK2 algorithm with a new spatial model and a refined method for dynamic suggestions. A controlled experiment shows HotStrokes effectively augments the existing "menu and hotkey" command activation paradigm. Results show the method is efficient by reducing command activation time by 43% compared to linear menus. The method is also easy to learn with a high adoption rate, replacing 91% of linear menu usage. Finally, combining linear menus, hotkeys, and HotStrokes leads to 24% faster command activation overall.

[1]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[2]  Kellogg S. Booth,et al.  Graphically enhanced keyboard accelerators for GUIs , 2010, Graphics Interface.

[3]  Pierre Dragicevic,et al.  Strategies for accelerating on-line learning of hotkeys , 2007, CHI.

[4]  Kasper Hornbæk,et al.  Vulture: a mid-air word-gesture keyboard , 2014, CHI.

[5]  Shumin Zhai,et al.  Foundational Issues in Touch-Surface Stroke Gesture Design - An Integrative Review , 2012, Found. Trends Hum. Comput. Interact..

[6]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[7]  V. S. Reed,et al.  Pictorial superiority effect. , 1976, Journal of experimental psychology. Human learning and memory.

[8]  Carl Gutwin,et al.  Promoting Hotkey use through rehearsal with ExposeHK , 2013, CHI.

[9]  Per Ola Kristensson,et al.  Investigating Tilt-based Gesture Keyboard Entry for Single-Handed Text Entry on Large Devices , 2017, CHI.

[10]  Yuanchun Shi,et al.  Tap, Dwell or Gesture?: Exploring Head-Based Text Entry Techniques for HMDs , 2017, CHI.

[11]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[12]  Iris van Rooij,et al.  Satisficing and the Use of Keyboard Shortcuts: Being Good Enough Is Enough? , 2013, Interact. Comput..

[13]  Ravin Balakrishnan,et al.  Simple vs. compound mark hierarchical marking menus , 2004, UIST '04.

[14]  Walter F. Bischof,et al.  Learning and performance with gesture guides , 2013, CHI.

[15]  Yang Li,et al.  Gesture avatar: a technique for operating mobile user interfaces using gestures , 2011, CHI.

[16]  Carl Gutwin,et al.  Dips and ceilings: understanding and supporting transitions to expertise in user interfaces , 2011, CHI.

[17]  Andy Cockburn,et al.  An empirical characterisation of electronic document navigation , 2008, Graphics Interface.

[18]  William Buxton,et al.  User learning and performance with marking menus , 1994, CHI '94.

[19]  Gilles Bailly,et al.  Flower menus: a new type of marking menu with large menu breadth, within groups and efficient expert mode memorization , 2008, AVI '08.

[20]  Abigail Sellen,et al.  An Empirical Evaluation of Some Articulatory and Cognitive Aspects of Marking Menus , 1993, Hum. Comput. Interact..

[21]  Shumin Zhai,et al.  Shorthand writing on stylus keyboard , 2003, CHI '03.

[22]  Olivier Chapuis,et al.  MarkPad: Augmenting Touchpads for Command Selection , 2017, CHI.

[23]  Xiaojun Bi,et al.  CommandBoard: Creating a General-Purpose Command Gesture Input Space for Soft Keyboard , 2017, UIST.

[24]  George W. Fitzmaurice,et al.  The Hotbox: efficient access to a large number of menu-items , 1999, CHI '99.

[25]  Patrick Baudisch,et al.  Imaginary interfaces: spatial interaction with empty hands and without visual feedback , 2010, UIST.

[26]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[27]  Olivier Bau,et al.  OctoPocus: a dynamic guide for learning gesture-based command sets , 2008, UIST '08.

[28]  David M. Lane,et al.  Hidden Costs of Graphical User Interfaces: Failure to Make the Transition from Menus and Icon Toolbars to Keyboard Shortcuts , 2005, Int. J. Hum. Comput. Interact..

[29]  Gilles Bailly,et al.  Wave Menus: Improving the Novice Mode of Hierarchical Marking Menus , 2007, INTERACT.

[30]  Shumin Zhai,et al.  Shapewriter on the iphone: from the laboratory to the real world , 2009, CHI Extended Abstracts.

[31]  Carl Gutwin,et al.  Supporting Novice to Expert Transitions in User Interfaces , 2014, ACM Comput. Surv..

[32]  Shumin Zhai,et al.  Command strokes with and without preview: using pen gestures on keyboard for command selection , 2007, CHI.

[33]  Nadine Mandran,et al.  Wavelet menus on handheld devices: stacking metaphor for novice mode and eyes-free selection for expert mode , 2010, AVI.

[34]  Claire C. Gordon,et al.  A Supplement to the 1995 Matched Anthropometric Database of U.S. Marine Corps Personnel: Summary Statistics , 1996 .

[35]  Gordon Kurtenbach,et al.  The design and evaluation of marking menus , 1993 .

[36]  Patrick Baudisch,et al.  Imaginary phone: learning imaginary interfaces by transferring spatial memory from a familiar device , 2011, UIST.

[37]  Shumin Zhai,et al.  Using strokes as command shortcuts: cognitive benefits and toolkit support , 2009, CHI.

[38]  Shumin Zhai,et al.  The word-gesture keyboard: reimagining keyboard interaction , 2012, CACM.

[39]  Yang Li.  Gesture search: a tool for fast mobile data access , 2010, UIST '10.

[40]  Daniel Vogel,et al.  Finger-Aware Shortcuts , 2016, CHI.

[41]  Edward Cutrell,et al.  Bimanual Interaction on the Microsoft Office Keyboard , 2003, INTERACT.

[42]  Daniel Vogel,et al.  FingerArc and FingerChord: Supporting Novice to Expert Transitions with Guided Finger-Aware Shortcuts , 2018, UIST.

[43]  Paul K. Wright,et al.  Toolglasses, Marking Menus, and Hotkeys: A Comparison of One and Two-Handed Command Selection Techniques , 2004, Graphics Interface.

[44]  Shumin Zhai,et al.  Bimanual gesture keyboard , 2012, UIST.

[45]  Gilles Bailly,et al.  Wavelet menus: a stacking metaphor for adapting marking menus to mobile devices , 2009, Mobile HCI.

[46]  Gilles Bailly,et al.  IconHK: Using Toolbar button Icons to Communicate Keyboard Shortcuts , 2017, CHI.

[47]  Kun Li,et al.  M3 Gesture Menu: Design and Experimental Analyses of Marking Menus for Touchscreen Mobile Interaction , 2018, CHI.

[48]  William Buxton,et al.  Contextual Animation of Gestural Commands , 1994, Comput. Graph. Forum.

[49]  Carl Gutwin,et al.  Investigating the Post-Training Persistence of Expert Interaction Techniques , 2017, ACM Trans. Comput. Hum. Interact..