Using strokes as command shortcuts: cognitive benefits and toolkit support

This paper investigates using stroke gestures as shortcuts to menu selection. We first experimentally measured the performance and ease of learning of stroke shortcuts in comparison to keyboard shortcuts when there is no mnemonic link between the shortcut and the command. While both types of shortcuts had the same level of performance with enough practice, stroke shortcuts had substantial cognitive advantages in learning and recall. With the same amount of practice, users could successfully recall more shortcuts and make fewer errors with stroke shortcuts than with keyboard shortcuts. The second half of the paper focuses on UI development support and articulates guidelines for toolkits to implement stroke shortcuts in a wide range of software applications. We illustrate how to apply these guidelines by introducing the Stroke Shortcuts Toolkit (SST) which is a library for adding stroke shortcuts to Java Swing applications with just a few lines of code.

[1]  Shumin Zhai,et al.  Modeling human performance of pen stroke gestures , 2007, CHI.

[2]  Shumin Zhai,et al.  Command strokes with and without preview: using pen gestures on keyboard for command selection , 2007, CHI.

[3]  Ravin Balakrishnan,et al.  Simple vs. compound mark hierarchical marking menus , 2004, UIST '04.

[4]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[5]  Catherine G. Wolf.  CAN PEOPLE USE GESTURE COMMANDS? , 1986, SGCH.

[6]  Charles C. Tappert,et al.  Cursive Script Recognition by Elastic Matching , 1982, IBM J. Res. Dev..

[7]  William Buxton,et al.  Contextual Animation of Gestural Commands , 1994, Comput. Graph. Forum.

[8]  Abigail Sellen,et al.  Two-handed input in a compound task , 1994, CHI 1994.

[9]  Paul K. Wright,et al.  Toolglasses, Marking Menus, and Hotkeys: A Comparison of One and Two-Handed Command Selection Techniques , 2004, Graphics Interface.

[10]  James A. Landay,et al.  Implications for a gesture design tool , 1999, CHI '99.

[11]  Desney S. Tan,et al.  InkSeine: In Situ search for active note taking , 2007, CHI.

[12]  William Buxton,et al.  User learning and performance with marking menus , 1994, CHI 1994.

[13]  Palmer Morrel-Samuels,et al.  Clarifying the Distinction Between Lexical and Gestural Commands , 1990, Int. J. Man Mach. Stud..

[14]  Yvonne Rogers,et al.  Evaluating the meaningfulness of icon sets to represent command operations , 1986 .

[15]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[16]  F. Craik,et al.  Levels of Pro-cessing: A Framework for Memory Research , 1975 .

[17]  Pierre Dragicevic,et al.  Strategies for accelerating on-line learning of hotkeys , 2007, CHI.

[18]  James A. Landay,et al.  SATIN: a toolkit for informal ink-based applications , 2000, UIST '00.

[19]  Olivier Bau,et al.  OctoPocus: a dynamic guide for learning gesture-based command sets , 2008, UIST '08.

[20]  George W. Fitzmaurice,et al.  The Hotbox: efficient access to a large number of menu-items , 1999, CHI '99.

[21]  David M. Lane,et al.  Hidden Costs of Graphical User Interfaces: Failure to Make the Transition from Menus and Icon Toolbars to Keyboard Shortcuts , 2005, Int. J. Hum. Comput. Interact..

[22]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[23]  James A. Landay,et al.  Visual similarity of pen gestures , 2000, CHI.

[24]  Timothy S. Miller,et al.  Fluid inking: augmenting the medium of free-form inking with gestures , 2006, Graphics Interface.

[25]  James R. Rhyne,et al.  The paper-like interface , 1989 .

[26]  Terry Winograd,et al.  Benefits of merging command selection and direct manipulation , 2005, TCHI.

[27]  Shumin Zhai,et al.  Shorthand writing on stylus keyboard , 2003, CHI '03.