FingerArc and FingerChord: Supporting Novice to Expert Transitions with Guided Finger-Aware Shortcuts

Keyboard shortcuts can be more efficient than graphical input, but they are underused by most users. To alleviate this, we present "Guided Finger-Aware Shortcuts" to reduce the gulf between graphical input and shortcut activation. The interaction technique works by recognising when a special hand posture is used to press a key, then allowing secondary finger movements to select among related shortcuts if desired. Novice users can learn the mappings through dynamic visual guidance revealed by holding a key down, but experts can trigger shortcuts directly without pausing. Two variations are described: FingerArc uses the angle of the thumb, and FingerChord uses a second key press. The techniques are motivated by an interview study identifying factors hindering the learning, use, and exploration of keyboard shortcuts. A controlled comparison with conventional keyboard shortcuts shows the techniques encourage overall shortcut usage, make interaction faster, less error-prone, and provide advantages over simply adding visual guidance to standard shortcuts.

[1]  Carl Gutwin,et al.  Testing the rehearsal hypothesis with two FastTap interfaces , 2015, Graphics Interface.

[2]  Gordon Kurtenbach,et al.  The design and evaluation of marking menus , 1993 .

[3]  Carl Gutwin,et al.  The usability of CommandMaps in realistic tasks , 2014, CHI.

[4]  Shumin Zhai,et al.  Using strokes as command shortcuts: cognitive benefits and toolkit support , 2009, CHI.

[5]  Olivier Chapuis,et al.  MarkPad: Augmenting Touchpads for Command Selection , 2017, CHI.

[6]  Shumin Zhai,et al.  The word-gesture keyboard: reimagining keyboard interaction , 2012, CACM.

[7]  Xiang Cao,et al.  Detecting and leveraging finger orientation for interaction with direct-touch surfaces , 2009, UIST '09.

[8]  Richard Alterman,et al.  Training towards mastery: overcoming the active user paradox , 2008, NordiCHI.

[9]  Manabu Hashimoto,et al.  Marker-less piano fingering recognition using sequential depth images , 2013, The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision.

[10]  Lawrence R. Ness,et al.  Are We There Yet - Data Saturation in Qualitative Research (TQR Published).pdf , 2015 .

[11]  Florian Alt,et al.  Extending Keyboard Shortcuts with Arm and Wrist Rotation Gestures , 2018, CHI.

[12]  Daniel Vogel,et al.  Characterizing Finger Pitch and Roll Orientation During Atomic Touch Actions , 2018, CHI.

[13]  Franklin P. Tamborello,et al.  Pros, Cons, and Changing Behavior: An Application in the Use of the Keyboard to Issue Commands , 2005 .

[14]  Jong Wook Kim,et al.  Learning, Forgetting, and Relearning for Keystroke- and Mouse-Driven Tasks: Relearning Is Important , 2015, Hum. Comput. Interact..

[15]  Feng Wang,et al.  Empirical evaluation for finger input properties in multi-touch interaction , 2009, CHI.

[16]  Steven Bathiche,et al.  A practical pressure sensitive computer keyboard , 2009, UIST '09.

[17]  Carl Gutwin,et al.  Testing the robustness and performance of spatially consistent interfaces , 2013, CHI.

[18]  Carl Gutwin,et al.  HandMark Menus: Rapid Command Selection and Large Command Sets on Multi-Touch Displays , 2016, CHI.

[19]  Olivier Bau,et al.  OctoPocus: a dynamic guide for learning gesture-based command sets , 2008, UIST '08.

[20]  Olivier Bau,et al.  Arpège: learning multitouch chord gestures vocabularies , 2013, ITS.

[21]  Carl Gutwin,et al.  Faster Command Selection on Touchscreen Watches , 2016, CHI.

[22]  William Buxton,et al.  User learning and performance with marking menus , 1994, CHI '94.

[23]  Xiang 'Anthony' Chen,et al.  The fat thumb: using the thumb's contact size for single-handed mobile interaction , 2012, Mobile HCI.

[24]  Jun Rekimoto,et al.  PreSense: interaction techniques for finger sensing input devices , 2003, UIST '03.

[25]  R. Bakeman.  Recommended effect size statistics for repeated measures designs , 2005, Behavior research methods.

[26]  Edward Cutrell,et al.  Bimanual Interaction on the Microsoft Office Keyboard , 2003, INTERACT.

[27]  Mary Beth Rosson,et al.  Paradox of the active user , 1987 .

[28]  J. Algina,et al.  Generalized eta and omega squared statistics: measures of effect size for some common research designs. , 2003, Psychological methods.

[29]  Chris Harrison,et al.  TapSense: enhancing finger interaction on touch surfaces , 2011, UIST.

[30]  Paul K. Wright,et al.  Toolglasses, Marking Menus, and Hotkeys: A Comparison of One and Two-Handed Command Selection Techniques , 2004, Graphics Interface.

[31]  Bonnie E. John,et al.  From sufficient to efficient usage: an analysis of strategic knowledge , 1997, CHI.

[32]  Shumin Zhai,et al.  Hard lessons: effort-inducing interfaces benefit spatial learning , 2007, CHI.

[33]  Gilles Bailly,et al.  Finger-count & radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces , 2010, CHI.

[34]  Carl Gutwin,et al.  Dips and ceilings: understanding and supporting transitions to expertise in user interfaces , 2011, CHI.

[35]  Shumin Zhai,et al.  Shorthand writing on stylus keyboard , 2003, CHI '03.

[36]  Carl Gutwin,et al.  Supporting Transitions to Expertise in Hidden Toolbars , 2016, CHI.

[37]  Kun Li,et al.  M3 Gesture Menu: Design and Experimental Analyses of Marking Menus for Touchscreen Mobile Interaction , 2018, CHI.

[38]  Yang Li,et al.  GestKeyboard: enabling gesture-based interaction on ordinary physical keyboard , 2014, CHI.

[39]  Daniel Vogel,et al.  Finger-Aware Shortcuts , 2016, CHI.

[40]  Heuristic Evaluation-A System Checklist , 2014 .

[41]  Jonna Häkkilä,et al.  Exploring finger specific touch screen interaction for mobile phone user interfaces , 2014, OZCHI.

[42]  Kellogg S. Booth,et al.  Graphically enhanced keyboard accelerators for GUIs , 2010, Graphics Interface.

[43]  Ravin Balakrishnan,et al.  Porous Interfaces for Small Screen Multitasking using Finger Identification , 2016, UIST.

[44]  Gilles Bailly,et al.  IconHK: Using Toolbar button Icons to Communicate Keyboard Shortcuts , 2017, CHI.

[45]  Shumin Zhai,et al.  Shapewriter on the iphone: from the laboratory to the real world , 2009, CHI Extended Abstracts.

[46]  Dmitry O. Gorodnichy,et al.  Detection and tracking of pianist hands and fingers , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[47]  Da-Yuan Huang,et al.  TouchSense: expanding touchscreen input vocabulary using different areas of users' finger pads , 2014, CHI.

[48]  Pierre Dragicevic,et al.  Strategies for accelerating on-line learning of hotkeys , 2007, CHI.

[49]  Carl Gutwin,et al.  Supporting Novice to Expert Transitions in User Interfaces , 2014, ACM Comput. Surv..

[50]  Geehyuk Lee,et al.  MelodicTap: fingering hotkey for touch tablets , 2016, OZCHI.

[51]  Bonnie E. John,et al.  The Strategic Use of Complex Computer Systems , 2000, Hum. Comput. Interact..

[52]  Carl Gutwin,et al.  Promoting Hotkey use through rehearsal with ExposeHK , 2013, CHI.

[53]  George W. Fitzmaurice,et al.  The Hotbox: efficient access to a large number of menu-items , 1999, CHI '99.

[54]  Jakob Nielsen,et al.  Measuring usability: preference vs. performance , 1994, CACM.

[55]  Judith S. Olson,et al.  The growth of software skill: a longitudinal look at learning & performance , 1993, INTERCHI.

[56]  Carl Gutwin,et al.  Improving command selection with CommandMaps , 2012, CHI.

[57]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[58]  Atsushi Sugiura,et al.  A user interface using fingerprint recognition: holding commands and data objects on fingers , 1998, UIST '98.

[59]  David M. Lane,et al.  Hidden Costs of Graphical User Interfaces: Failure to Make the Transition from Menus and Icon Toolbars to Keyboard Shortcuts , 2005, Int. J. Hum. Comput. Interact..

[60]  Ian H. Witten,et al.  On Frequency-Based Menu-Splitting Algorithms , 1984, Int. J. Man Mach. Stud..

[61]  Abigail Sellen,et al.  An Empirical Evaluation of Some Articulatory and Cognitive Aspects of Marking Menus , 1993, Hum. Comput. Interact..

[62]  Carl Gutwin,et al.  Single-Handed HandMark Menus: Rapid Command Selection on Tablets , 2016, ISS.

[63]  Mary Beth Rosson,et al.  Patterns of experience in text editing , 1983, CHI '83.

[64]  Caroline Appert,et al.  Multi-touch gestures for discrete and continuous control , 2014, AVI.

[65]  William Buxton,et al.  Contextual Animation of Gestural Commands , 1994, Comput. Graph. Forum.

[66]  Carl Gutwin,et al.  Investigating the Post-Training Persistence of Expert Interaction Techniques , 2017, ACM Trans. Comput. Hum. Interact..

[67]  Walter F. Bischof,et al.  Learning and performance with gesture guides , 2013, CHI.

[68]  Chris Harrison,et al.  Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction , 2012, CHI.

[69]  Daniel Vogel,et al.  Pinch-to-zoom-plus: an enhanced pinch-to-zoom that reduces clutching and panning , 2014, UIST.

[70]  D. C. Shapiro,et al.  Summary knowledge of results for skill acquisition: support for the guidance hypothesis. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[71]  Carl Gutwin,et al.  Multiplexing spatial memory: increasing the capacity of FastTap menus with multiple tabs , 2018, MobileHCI.

[72]  Hemant Bhaskar Surale,et al.  Experimental Analysis of Mode Switching Techniques in Touch-based User Interfaces , 2017, CHI.

[73]  Wendy E. Mackay,et al.  Fieldward and Pathward: Dynamic Guides for Defining Your Own Gestures , 2017, CHI.

[74]  Daniel Vogel,et al.  The Performance and Preference of Different Fingers and Chords for Pointing, Dragging, and Object Transformation , 2016, CHI.

[75]  Carl Gutwin,et al.  Rapid Command Selection on Multi-Touch Tablets with Single-Handed HandMark Menus , 2016, ISS.

[76]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[77]  Gilles Bailly,et al.  CtrlMouse et TouchCtrl: duplicating mode delimiters on the mouse , 2014, IHM.

[78]  D. Norman.  The Design of Everyday Things: Revised and Expanded Edition , 2013 .

[79]  Carl Gutwin,et al.  Faster command selection on tablets with FastTap , 2014, CHI.

[80]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[81]  Daniel Vogel,et al.  Adoiraccourcix: multi-touch command selection using finger identification , 2014, IHM.

[82]  John F. Canny,et al.  FingerSense: augmenting expressiveness to physical pushing button by fingertip identification , 2004, CHI EA '04.

[83]  Daniel Vogel,et al.  Leveraging finger identification to integrate multi-touch command selection and parameter manipulation , 2017, Int. J. Hum. Comput. Stud..

[84]  Brian D. Ehret,et al.  Learning where to look: location learning in graphical user interfaces , 2002, CHI.

[85]  Géry Casiez,et al.  WhichFingers: Identifying Fingers on Touch Surfaces and Keyboards using Vibration Sensors , 2017, UIST.

[86]  Iris van Rooij,et al.  Satisficing and the Use of Keyboard Shortcuts: Being Good Enough Is Enough? , 2013, Interact. Comput..

[87]  Gilles Bailly,et al.  Effects of Frequency Distribution on Linear Menu Performance , 2017, CHI.