Comparing Free Hand Menu Techniques for Distant Displays Using Linear, Marking and Finger-Count Menus

Distant displays such as interactive public displays (IPD) or interactive television (ITV) require new interaction techniques as traditional input devices may be limited or missing in these contexts. Free hand interaction, as sensed with computer vision techniques, presents a promising interaction technique. This paper presents the adaptation of three menu techniques for free hand interaction: Linear menu, Marking menu and Finger-Count menu. The first study based on a Wizard-of-Oz protocol focuses on Finger-Counting postures in front of interactive television and public displays. It reveals that participants do not choose the most efficient gestures neither before nor after the experiment. Results are used to develop a Finger-Count recognizer. The second experiment shows that all techniques achieve satisfactory accuracy. It also shows that Finger-Count requires more mental demand than other techniques.

[1]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[2]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[3]  Mike Wu,et al.  Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays , 2003, UIST '03.

[4]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[5]  Lars Bretzner,et al.  Computer Vision Based Recognition of Hand Gestures for Human-Computer Interaction , 2002 .

[6]  Antti Oulasvirta,et al.  It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre , 2008, CHI.

[7]  Joris IJsselmuiden,et al.  Extending touch: towards interaction with large-scale surfaces , 2009, ITS '09.

[8]  A BoltRichard,et al.  Put-that-there , 1980 .

[9]  Ravin Balakrishnan,et al.  Simple vs. compound mark hierarchical marking menus , 2004, UIST '04.

[10]  Sriram Subramanian,et al.  GesText: accelerometer-based gestural text-entry systems , 2010, CHI.

[11]  Human-Computer Interaction – INTERACT 2009 , 2009, Lecture Notes in Computer Science.

[12]  William Buxton,et al.  The limits of expert performance using hierarchic marking menus , 1993, INTERCHI.

[13]  Lars Bretzner,et al.  Using marking menus to develop command sets for computer vision based hand gesture interfaces , 2002, NordiCHI '02.

[14]  Pablo César,et al.  The Evolution of TV Systems, Content, and Users Toward Interactivity , 2009, Found. Trends Hum. Comput. Interact..

[15]  Philippe A. Palanque,et al.  Human-Computer Interaction – INTERACT 2007 , 2007, Lecture Notes in Computer Science.

[16]  Abhishek Ranjan,et al.  Interacting with large displays from a distance with vision-tracked multi-finger gestural input , 2005, SIGGRAPH '06.

[17]  Gilles Bailly,et al.  Leaf Menus: Linear Menus with Stroke Shortcuts for Small Handheld Devices , 2009, INTERACT.

[18]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Gilles Bailly,et al.  Wave Menus: Improving the Novice Mode of Hierarchical Marking Menus , 2007, INTERACT.

[20]  Raimund Dachselt,et al.  Three-dimensional menus: A survey and taxonomy , 2007, Comput. Graph..

[21]  Carl Gutwin,et al.  A predictive model of menu performance , 2007, CHI.

[22]  Gilles Bailly,et al.  Finger-count & radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces , 2010, CHI.

[23]  Nadine Mandran,et al.  Wavelet menus on handheld devices: stacking metaphor for novice mode and eyes-free selection for expert mode , 2010, AVI.

[24]  Hrvoje Benko,et al.  Pinch-the-sky dome: freehand multi-point interactions with immersive omni-directional data , 2010, CHI EA '10.

[25]  Daniel Vogel,et al.  Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users , 2004, UIST '04.

[26]  M. Weiser,et al.  An empirical comparison of pie vs. linear menus , 1988, CHI '88.

[27]  William Buxton,et al.  User learning and performance with marking menus , 1994, CHI '94.

[28]  Andy Cockburn,et al.  AAMU: adaptive activation area menus for improving selection in cascading pull-down menus , 2008, CHI.

[29]  Tovi Grossman,et al.  The design and evaluation of multitouch marking menus , 2010, CHI.

[30]  Jörg Müller,et al.  The Audience Funnel: Observations of Gesture Based Interaction With Multiple Large Displays in a City Center , 2011, Int. J. Hum. Comput. Interact..

[31]  Michel Beaudouin-Lafon,et al.  Charade: remote control of objects using free-hand gestures , 1993, CACM.

[32]  Albrecht Schmidt,et al.  Audience behavior around large interactive cylindrical screens , 2011, CHI.