Exploring the usefulness of finger-based 3D gesture menu selection

Counting using one's fingers is a potentially intuitive way to enumerate a list of items and lends itself naturally to gesture-based menu systems. In this paper, we present the results of the first comprehensive study on Finger-Count menus to investigate its usefulness as a viable option for 3D menu selection tasks. Our study compares 3D gesture-based finger counting (Finger Count menus) with two gesture-based menu selection techniques (Hand-n-Hold, Thumbs-Up), derived from existing motion-controlled video game menu selection strategies, as well as 3D Marking menus. We examined selection time, selection accuracy and user preference for all techniques. We also examined the impact of different spatial layouts for menu items and different menu depths. Our results indicate that Finger-Count menus are significantly faster than the other menu techniques we tested and are the most liked by participants. Additionally, we found that while Finger-Count menus and 3D Marking menus have similar selection accuracy, Finger-Count menus are almost twice as fast compared to 3D Marking menus.

[1]  Jörg Müller,et al.  Comparing Free Hand Menu Techniques for Distant Displays Using Linear, Marking and Finger-Count Menus , 2011, INTERACT.

[2]  Mathias Kölsch,et al.  Robust hand detection , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[3]  Terry Winograd,et al.  FlowMenu: combining command, text, and data entry , 2000, UIST '00.

[4]  Junsong Yuan,et al.  Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera , 2011, ACM Multimedia.

[5]  Daniel James Ryan,et al.  Finger and gesture recognition with Microsoft Kinect , 2012 .

[6]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[7]  J. I.  The Design of Experiments , 1936, Nature.

[8]  C. Jennings,et al.  Robust finger tracking with multiple cameras , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[9]  Colin Ware,et al.  A study of haptic linear and pie menus in a 3D fish tank VR environment , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[10]  Joseph J. LaViola,et al.  Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[11]  Doug A. Bowman,et al.  Tech-note: rapMenu: Remote Menu Selection Using Freehand Gestural Input , 2008, 2008 IEEE Symposium on 3D User Interfaces.

[12]  Shweta Kapur,et al.  Age-related changes in the control of finger force vectors. , 2010, Journal of applied physiology.

[13]  Ravin Balakrishnan,et al.  Simple vs. compound mark hierarchical marking menus , 2004, UIST '04.

[14]  Gilles Bailly,et al.  Finger-count & radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces , 2010, CHI.

[15]  Mark Green,et al.  JDCAD: A highly interactive 3D modeling system , 1994, Comput. Graph..

[16]  Joseph J. LaViola,et al.  An exploration of menu techniques using a 3D game input device , 2009, FDG.

[17]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[18]  M. Weiser,et al.  An empirical comparison of pie vs. linear menus , 1988, CHI '88.

[19]  Gordon Kurtenbach,et al.  The design and evaluation of marking menus , 1993 .

[20]  Doug A. Bowman,et al.  Design and evaluation of menu systems for immersive virtual environments , 2001, Proceedings IEEE Virtual Reality 2001.

[21]  Dominique Bechmann,et al.  The spin menu: a menu system for virtual environments , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[22]  Gang Ren,et al.  3D Marking menu selection with freehand gestures , 2012, 2012 IEEE Symposium on 3D User Interfaces (3DUI).

[23]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[24]  Thiago R. Trigo,et al.  An Analysis of Features for Hand-Gesture Classification , 2010 .