Interaction with hand gesture for a back-projection wall

(Semi)immersive displays like back projection walls have become widely used in various visual domains. Despite the superior image quality of `through-the-windows VR' users are still not satisfied with the interaction capabilities - mostly due to the extra wiring for the interaction devices. This paper represents a hand gesture interaction system for a back-projection wall environment, supporting object manipulation (translation, rotation, and uniform scale) and selection tasks through the vision-tracking technique. We propose to use thimble-shaped fingertip markers made of white printing paper with a `black light' source. Gesture parameters are calculated by 3D positions of the marked fingertips and their pinching states

[1]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[2]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[3]  Paul A. Beardsley,et al.  Computer Vision for Interactive Computer Graphics , 1998, IEEE Computer Graphics and Applications.

[4]  Christian Leubner,et al.  Computer-vision-based human-computer interaction with a back projection wall using arm gestures , 2001, Proceedings 27th EUROMICRO Conference. 2001: A Net Odyssey.

[5]  P. Costello.  Health and Safety Issues associated with Virtual Reality-A Review of Current Literature , 1997 .

[6]  James L. Crowley,et al.  Finger Tracking as an Input Device for Augmented Reality , 1995 .

[7]  Christopher R. Dance,et al.  Binarising camera images for OCR , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[8]  Yoichi Sato,et al.  Vision-Based Face Tracking System for Large Displays , 2002, UbiComp.

[9]  Dieter Schmalstieg,et al.  Finger tracking for interaction in augmented environments , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[10]  Mark R. Mine,et al.  Virtual Environment Interaction Techniques , 1995 .

[11]  Kazuhiko Takahashi,et al.  Real-time estimation of human body postures using Kalman filter , 1999, 8th IEEE International Workshop on Robot and Human Interaction. RO-MAN '99 (Cat. No.99TH8483).

[12]  François Bérard,et al.  Bare-hand human-computer interaction , 2001, PUI '01.

[13]  Markus Kohler,et al.  Using the Kalman Filter to track Human Interactive Motion - Modelling and Initialization of the Kalm , 1997 .

[14]  Yoichi Sato,et al.  Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[15]  Tony Lindeberg,et al.  Use your hand as a 3D mouse , 1998 .

[16]  J. Rekimoto,et al.  Perceptual Surfaces : Towards a Human and Object Sensitive Interactive Display , 1997 .

[17]  Yoichi Sato,et al.  Real-time input of 3D pose and gestures of a user's hand and its applications for HCI , 2001, Proceedings IEEE Virtual Reality 2001.

[18]  William Ribarsky,et al.  Interaction in Semi-Immersiv e Large Display Environments , 1998 .

[19]  Yoichi Sato,et al.  Fast tracking of hands and fingertips in infrared images for augmented desk interface , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[20]  Jean-Baptiste de la Rivière,et al.  Hand Postures Recognition in Large - Display VR Environments , 2003, Gesture Workshop.

[21]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Roberto Cipolla,et al.  Robust structure from motion using motion parallax , 1993, 1993 (4th) International Conference on Computer Vision.

[23]  Thomas S. Huang,et al.  Gesture modeling and recognition using finite state machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[24]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[25]  Stan Sclaroff,et al.  3D hand pose reconstruction using specialized mappings , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[27]  William Ribarsky,et al.  T owards Spontaneous Interaction with the Perceptive Workbench, a Semi-Immersive Virtual Environment* , 2000 .

[28]  Wayne Niblack,et al.  An introduction to digital image processing , 1986 .

[29]  Ying Wu,et al.  Modeling the constraints of human hand motion , 2000, Proceedings Workshop on Human Motion.