Gesture select:: acquiring remote targets on large displays without pointing

When working at a large wall display, even if partially utilized, many targets are likely to be distant from the user, requiring walking, which is slow, and interrupts workflow. We propose a novel technique for selecting remote targets called Gesture Select, in which users draw an initial mark, in a target's direction; rectilinear gestures represented as icons are dynamically overlaid on targets within a region of interest; the user then continues by drawing the continuation mark corresponding to the target, to select it. Extensions to this technique to support working with remote content for an extended period, and learning gesture shortcuts are presented. A formal experiment indicates Gesture Select significantly outperformed direct selection for mid/far targets. Further analysis suggests Gesture Select performance is principally affected by the extent to which users can read the gestures, influenced by distance and perspective warping, and the gesture complexity in the ROI. The results of a second 2-D experiment with labeled targets indicate Gesture Select significantly outperformed direct selection and an existing technique.

[1]  Thomas P. Moran,et al.  Tivoli: an electronic whiteboard for informal workgroup meetings , 1993, INTERCHI.

[2]  Anastasia Bezerianos,et al.  The vacuum: facilitating the manipulation of distant objects , 2005, CHI.

[3]  Desney S. Tan,et al.  WinCuts: manipulating arbitrary window regions for more effective use of screen space , 2004, CHI EA '04.

[4]  Timothy S. Miller,et al.  Fluid inking: augmenting the medium of free-form inking with gestures , 2006, Graphics Interface.

[5]  Carl Gutwin,et al.  Improving selection of off-screen targets with hopping , 2006, CHI.

[6]  Anastasia Bezerianos,et al.  View and space management on large displays , 2005, IEEE Computer Graphics and Applications.

[7]  Carl Gutwin,et al.  Supporting multi-point interaction in visual workspaces , 2007, CHI.

[8]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[9]  Takeo Igarashi,et al.  Flatland: new dimensions in office whiteboards , 1999, CHI '99.

[10]  P. Fitts The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[11]  Carl Gutwin,et al.  Bubble radar: efficient pen-based interaction , 2006, AVI '06.

[12]  Mary Czerwinski,et al.  Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-Operated Systems , 2003, INTERACT.

[13]  Chris Harrison,et al.  Providing dynamically changeable physical buttons on a visual display , 2009, CHI.

[14]  Pat Hanrahan,et al.  A distributed graphics system for large tiled displays , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[15]  Ravin Balakrishnan,et al.  Simple vs. compound mark hierarchical marking menus , 2004, UIST '04.

[16]  S. H. Sato,et al.  Interaction design for large displays , 1997, INTR.

[17]  George W. Fitzmaurice,et al.  A remote control interface for large displays , 2004, UIST '04.

[18]  Richard May,et al.  A Survey of Large High-Resolution Display Technologies, Techniques, and Applications , 2006, IEEE Virtual Reality Conference (VR 2006).

[19]  Takeo Igarashi,et al.  Ninja cursors: using multiple cursors to assist target acquisition on large screens , 2008, CHI.

[20]  Terry Winograd,et al.  FlowMenu: combining command, text, and data entry , 2000, UIST '00.

[21]  Terry Winograd,et al.  Fluid interaction with high-resolution wall-size displays , 2001, UIST '01.

[22]  Saul Greenberg,et al.  Speech-filtered bubble ray: improving target acquisition on display walls , 2007, ICMI '07.

[23]  Mark W. Newman,et al.  Escape: a target selection technique using visually-cued gestures , 2008, CHI.

[24]  William Buxton,et al.  The limits of expert performance using hierarchic marking menus , 1993, INTERCHI.

[25]  Carl Gutwin,et al.  Superflick: a natural and efficient technique for long-distance object placement on digital tables , 2006, Graphics Interface.

[26]  Olivier Bau,et al.  OctoPocus: a dynamic guide for learning gesture-based command sets , 2008, UIST '08.

[27]  Patrick Baudisch,et al.  Improving drag-and-drop on wall-size displays , 2005, Graphics Interface.

[28]  Abhishek Ranjan,et al.  Interacting with large displays from a distance with vision-tracked multi-finger gestural input , 2005, SIGGRAPH '06.

[29]  Patrick Baudisch,et al.  Stitching: pen gestures that span multiple displays , 2004, AVI.

[30]  Patrick Baudisch,et al.  Design and analysis of delimiters for selection-action pen gesture phrases in scriboli , 2005, CHI.

[31]  Joseph J. LaViola,et al.  GestureBar: improving the approachability of gesture-based interfaces , 2009, CHI.

[32]  Daniel Vogel,et al.  Distant freehand pointing and clicking on very large, high resolution displays , 2005, UIST.

[33]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[34]  Jean-Daniel Fekete,et al.  Motion-pointing: target selection using elliptical motions , 2009, CHI.