CUDA-based real-time hand gesture interaction and visualization for CT volume dataset using leap motion

Touchless interaction has received considerable attention in recent years with benefit of removing barriers of physical contact. Several approaches are available to achieve mid-air interactions. However, most of these techniques cause discomfort when the interaction method is not direct manipulation. In this paper, gestures based on unimanual and bimanual interactions with different tools for exploring CT volume dataset are designed to perform the similar tasks in realistic applications. Focus + context approach based on GPU volume ray casting by trapezoid-shaped transfer function is used for visualization and the level-of-detail technique is adopted for accelerating interactive rendering. Comparing the effectiveness and intuitiveness of interaction approach with others by experiments, ours has a better performance and superiority with less completion time. Moreover, the bimanual interaction with more advantages is timesaving when performing continuous exploration task.

[1]  Xiaoru Yuan,et al.  WYSIWYG (What You See is What You Get) Volume Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[2]  Pere-Pau Vázquez,et al.  The ViRVIG institute , 2011 .

[3]  Enrico Gobbetti,et al.  A single-pass GPU ray casting framework for interactive out-of-core rendering of massive volumetric datasets , 2008, The Visual Computer.

[4]  Shumin Zhai,et al.  Manual and cognitive benefits of two-handed input: an experimental study , 1998, TCHI.

[5]  Pere-Pau Vázquez,et al.  Depth-Enhanced Maximum Intensity Projection , 2010, VG@Eurographics.

[6]  Orit Shaer,et al.  Reality-based interaction: a framework for post-WIMP interfaces , 2008, CHI.

[7]  Pere-Pau Vázquez,et al.  The virtual magic lantern: an interaction metaphor for enhanced medical data inspection , 2009, VRST '09.

[8]  B. Laha,et al.  Design of the BareHand Volume Cracker for Analysis of Raw Volumetric Data , 2014 .

[9]  J. A. Bærentzen,et al.  PorkCAD: Case study of the design of a pork product prototyper , 2013 .

[10]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[11]  Stefan Bruckner,et al.  TECHNICAL REPORT VolumeShop: An Interactive System for Direct Volume , 2022 .

[12]  Doug A. Bowman,et al.  Effects of Immersion on Visual Analysis of Volume Data , 2013, IEEE Trans. Vis. Comput. Graph..

[13]  Daniel J. Wigdor,et al.  Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces , 2008, AVI '08.

[14]  Enrico Gobbetti,et al.  COVRA: A compression‐domain output‐sensitive volume rendering architecture based on a sparse representation of voxel blocks , 2012, Comput. Graph. Forum.

[15]  Kenton O'Hara,et al.  Touchless interaction in surgery , 2014, CACM.

[16]  Barr,et al.  Superquadrics and Angle-Preserving Transformations , 1981, IEEE Computer Graphics and Applications.

[17]  Dennis Wixon,et al.  The State-Transition Model of Input , 2011 .

[18]  Thomas Pederson,et al.  Touch-less interaction with medical images using hand & foot gestures , 2013, UbiComp.

[19]  William Buxton,et al.  A three-state model of graphical input , 1990, INTERACT.

[20]  Marc Levoy,et al.  Display of surfaces from volume data , 1988, IEEE Computer Graphics and Applications.

[21]  Jeremy R. Cooperstock,et al.  Did "Minority Report" Get It Wrong? Superiority of the Mouse over 3D Input Devices in a 3D Placement Task , 2009, INTERACT.

[22]  Jovan Popovic,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH '09.

[23]  Stefan Bruckner,et al.  Style Transfer Functions for Illustrative Volume Rendering , 2007, Comput. Graph. Forum.

[24]  Charles L. A. Clarke,et al.  Bimanual and unimanual image alignment: an evaluation of mouse-based techniques , 2005, UIST '05.

[25]  Karin Coninx,et al.  Exploring the Effects of Environment Density and Target Visibility on Object Selection in 3D Virtual Environments , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[26]  Lisa Gralewski,et al.  Theory and Practice of Computer Graphics , 2004 .

[27]  Peng Song,et al.  A handle bar metaphor for virtual object manipulation with mid-air interaction , 2012, CHI.

[28]  William Buxton,et al.  When it gets more difficult, use both hands: exploring bimanual curve manipulation , 2005, Graphics Interface.

[29]  Luigi Gallo.  A study on the degrees of freedom in touchless interaction , 2013, SA '13.

[30]  Qi Zhang,et al.  Rapid scalar value classification and volume clipping for interactive 3D medical image visualization , 2010, The Visual Computer.

[31]  Timo Ropinski,et al.  Tentative Results in Focus-Based Medical Volume Visualization , 2005, Smart Graphics.

[32]  Ken Brodlie,et al.  Focus and context for volume visualization , 2004, Proceedings Theory and Practice of Computer Graphics, 2004..

[33]  Yanlin Luo,et al.  Distance-Based Focus + Context Models for Exploring Large Volumetric Medical Datasets , 2012, Computing in Science & Engineering.

[34]  Tim McInerney,et al.  HingeSlicer: interactive exploration of volume images using extended 3D slice plane widgets , 2006, Graphics Interface.

[35]  Tovi Grossman,et al.  Multi-finger gestural interaction with 3d volumetric displays , 2004, UIST '04.

[36]  Wang Hui-nan.  Multi-Finger Gestural Interaction with 3D Volumetric Displays , 2008 .

[37]  Thomas Knott,et al.  Influence of the bimanual frame of reference with haptics for unimanual interaction tasks in virtual environments , 2011, 2011 IEEE Symposium on 3D User Interfaces (3DUI).