A Practical Vision Based Approach to Unencumbered Direct Spatial Manipulation in Virtual Worlds

We present a practical approach for developing interactive environments that allows humans to interact with large complex 3D models without them having to manually operate input devices. The system provides support for scene manipulation based on hand tracking and gesture recognition and for direct 3D interaction with the 3D models in the display space if a suitably registered 3D display is used. Being based on markerless tracking of a user’s two hands, the system does not require users to wear any input or output devices. 6DOF input is provided by using both hands simultaneously, making the tracker more robust since only tracking of position information is required. The effectiveness of the method is demonstrated with a simple application for model manipulation on a large stereo display, in which rendering constraints are met by employing state-of-the-art multiresolution techniques.

[1]  Ramon Mas,et al.  Hand Tracking and Gesture Recognition for Human-Computer Interaction , 2005 .

[2]  Paolo Cignoni,et al.  Adaptive TetraPuzzles - Efficient Out-of-core Construction and Visualization of Gigantic Polygonal Models , 2004 .

[3]  Nicol N. Schraudolph,et al.  3D hand tracking by rapid stochastic gradient descent using a skinning model , 2004 .

[4]  Tieniu Tan,et al.  Real time hand tracking by combining particle filtering and mean shift , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[5]  Abdesselam Bouzerdoum,et al.  Skin segmentation using color pixel classification: analysis and comparison , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Francois Blais,et al.  GoLD: Interactive Display of Huge Colored and Textured Models Louis Borgeat Guy Godin Franc ‚ois Blais Philippe Massicotte National Research Council of Canada , 2005, SIGGRAPH 2005.

[7]  Patrick Lemoine,et al.  Interaction Techniques: 3D Menus-based Paradigm , 2003 .

[8]  Stan Sclaroff,et al.  Estimation and prediction of evolving color distributions for skin segmentation under varying illumination , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Shan Lu,et al.  Color-based hands tracking system for sign language recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Oliver Schreer,et al.  Vision-based skin-colour segmentation of moving hands for real-time applications , 2004 .

[11]  Andrea Giachetti,et al.  Matching techniques to compute image motion , 2000, Image Vis. Comput..

[12]  Cristina Manresa-Yee,et al.  Hand Tracking and Gesture Recognition for Human-Computer Interaction , 2009, Progress in Computer Vision and Image Analysis.

[13]  Dante Augusto Couto Barone,et al.  Performance evaluation of single and multiple-Gaussian models for skin color modeling , 2002, Proceedings. XV Brazilian Symposium on Computer Graphics and Image Processing.

[14]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[15]  Paulo R. S. Mendonça,et al.  Model-Based Hand Tracking Using an Unscented Kalman Filter , 2001, BMVC.

[16]  Stan Sclaroff,et al.  Automatic 2D Hand Tracking in Video Sequences , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[17]  Li-Qun Xu.  Simultaneous tracking and segmentation of two free moving hands in a video conferencing scenario , 2003 .

[18]  Paolo Cignoni,et al.  Batched multi triangulation , 2005, VIS 05. IEEE Visualization, 2005..

[19]  Francis Quek,et al.  Comparison of five color models in skin pixel classification , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[20]  Dinesh Manocha,et al.  Quick-VDR: interactive view-dependent rendering of massive models , 2004, IEEE Visualization 2004.

[21]  Raimondo Schettini,et al.  Skin segmentation using multiple thresholding , 2006, Electronic Imaging.

[22]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[23]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..