3D selection with freehand gesture

The use of 3D computer graphics is important in a very wide range of applications. However, user interaction with 3D applications is still challenging and often does not lend itself to established techniques that have been developed primarily for 2D desktop interaction. Meanwhile, 3D user interfaces that rely on tracking hand-held devices or fiducial markers attached to the user are cumbersome or entirely inappropriate in some situations. These challenges may be addressed by refining and building on the increasing use of freehand gestural input, i.e. without markers or hand-held devices, to extend the fluidity and immediacy of today's 2D touch-based interactions. In this paper, we analyze the characteristics of freehand gestural 3D interaction, and report a set of 3 related evaluation studies focused on the fundamental user interface task of object selection. We found that interaction design requiring a high accuracy single action are not appropriate for freehand gestural selection, while separating it into several connected low demand operations could be a potential solution; that our Reach technique is accurate and potentially useful for option selection tasks with freehand gesture; and that strong directional effects influence performance and usability of both 2D and 3D option selection. We propose guidelines for designers of 3D freehand gestural interaction based on our evaluation results.