Mid-air interactions above stereoscopic interactive tables

Stereoscopic tabletops offer unique visualization capabilities, enabling users to perceive virtual objects as if they were lying above the surface. While allowing virtual objects to coexist with user actions in the physical world, interaction with these virtual objects above the surface presents interesting challenges. In this paper, we aim to understand which approaches to 3D virtual object manipulations are suited to this scenario. To this end, we implemented five different techniques based on the literature. Four are mid-air techniques, while the remainder relies on multi-touch gestures, which act as a baseline. Our setup combines affordable non-intrusive tracking technologies with a multi-touch stereo tabletop, providing head and hands tracking, to improve both depth perception and seamless interactions above the table. We conducted a user evaluation to find out which technique appealed most to participants. Results suggest that mid-air interactions, combining direct manipulation with six degrees of freedom for the dominant hand, are both more satisfying and efficient than the alternatives tested.

[1]  Laurent Grisoni,et al.  The effect of DOF separation in 3D manipulation tasks with multi-touch displays , 2010, VRST '10.

[2]  M. Sheelagh T. Carpendale,et al.  Rotation and translation mechanisms for tabletop interaction , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[3]  Saul Greenberg,et al.  The Continuous Interaction Space: Interaction Techniques Unifying Touch and Gesture on and above a Digital Surface , 2011, INTERACT.

[4]  Gerd Bruder,et al.  Effects of visual conflicts on 3D selection task performance in stereoscopic display environments , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[5]  Jane Yung-jen Hsu,et al.  Touching the void: direct-touch interaction for intangible displays , 2010, CHI.

[6]  Martin Hachet,et al.  Beyond the mouse: Understanding user gestures for manipulating 3D objects from touchscreen inputs , 2012, Comput. Graph..

[7]  Steven K. Feiner,et al.  Balloon Selection: A Multi-Finger Technique for Accurate Low-Fatigue 3D Selection , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[8]  Joaquim A. Jorge,et al.  Mockup Builder: 3D modeling on and above the surface , 2013, Comput. Graph..

[9]  Sylvain Paris,et al.  6D hands: markerless hand-tracking for computer aided design , 2011, UIST.

[10]  Andreas Butz,et al.  Interactions in the air: adding further depth to interactive tabletops , 2009, UIST '09.

[11]  Stephanie Houde,et al.  Iterative design of an interface for easy 3-D direct manipulation , 1992, CHI.

[12]  Jan O. Borchers,et al.  Direct manipulation and the third dimension: co-planar dragging on 3d displays , 2012, ITS '12.

[13]  Klaus H. Hinrichs,et al.  Triangle cursor: interactions with objects above the tabletop , 2011, ITS '11.

[14]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[15]  Martin Hachet,et al.  Toucheo: multitouch and stereo combined in a seamless workspace , 2011, UIST.

[16]  M. Sheelagh T. Carpendale,et al.  Shallow-depth 3d interaction: design and evaluation of one-, two- and three-touch techniques , 2007, CHI.

[17]  Philip L. Davidson,et al.  A screen-space formulation for 2D and 3D direct manipulation , 2009, UIST '09.

[18]  Martin Hachet,et al.  tBox: a 3d transformation widget designed for touch-screens , 2011, CHI.

[19]  Jovan Popovic,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH '09.

[20]  Joaquim A. Jorge,et al.  Mockup builder: direct 3D modeling on and above the surface in a continuous interaction space , 2012, Graphics Interface.

[21]  Pedro Lopes,et al.  Hands-on interactive tabletop LEGO application , 2011, Advances in Computer Entertainment Technology.

[22]  Patrick Baudisch,et al.  Separability of spatial manipulations in multi-touch interfaces , 2009, Graphics Interface.

[23]  M. Sheelagh T. Carpendale,et al.  Sticky tools: full 6DOF force-based interaction for multi-touch tables , 2009, ITS '09.

[24]  Pedro Lopes,et al.  Combining bimanual manipulation and pen-based input for 3D modelling , 2011, SBIM '11.

[25]  Martin Hachet,et al.  Multi-touch RST in 2D and 3D spaces: Studying the impact of directness on user performance , 2011, 2011 IEEE Symposium on 3D User Interfaces (3DUI).

[26]  Peng Song,et al.  A handle bar metaphor for virtual object manipulation with mid-air interaction , 2012, CHI.

[27]  Antonio Krüger,et al.  Balloon selection revisited: multi-touch selection techniques for stereoscopic data , 2012, AVI.

[28]  David Kim,et al.  HoloDesk: direct 3d interactions with a situated see-through display , 2012, CHI.

[29]  Marc Alexa,et al.  The effect of perspective projection in multi-touch 3D interaction , 2012, Graphics Interface.

[30]  Laurent Grisoni,et al.  The design and evaluation of 3D positioning techniques for multi-touch displays , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).