Shadow gestures: 3D hand pose estimation using a single camera

This paper describes a system that uses a camera and a point light source to track a user's hand in three dimensions. Using depth cues obtained from projections of the hand and its shadow, the system computes the 3D position and orientation of two fingers (thumb and pointing finger). The system recognizes one dynamic and two static gestures. Recognition and pose estimation are user independent and robust. The system operates at the rate of 60 Hz and can be used as an intuitive input interface to applications that require multi-dimensional control. Examples include 3D fly-thru's, object manipulation and computer games.

[1]  Markus Kohler.  System architecture and techniques for gesture recognition in unconstrained environments , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[2]  Lei Wang,et al.  Shape from active shadow motion , 1993, Other Conferences.

[3]  Kenneth A. Loparo,et al.  Reconstruction of three-dimensional surfaces from two-dimensional binary images , 1989, IEEE Trans. Robotics Autom..

[4]  R. J. KohlerInformatik.  System Architecture and Techniques for Gesture Recognition in Unconstraint Environments , 1997 .

[5]  Takeo Kanade,et al.  Using shadows in finding surface orientations , 1983, Comput. Vis. Graph. Image Process..

[6]  Pierre Wellner,et al.  The DigitalDesk Calculator: Tangible Manipulation on , 1991 .

[7]  Ramakant Nevatia,et al.  Building Detection and Description from a Single Intensity Image , 1998, Comput. Vis. Image Underst..

[8]  Hiroaki Nishino,et al.  Interactive two-handed gesture interface in 3D virtual environments , 1997, VRST '97.

[9]  John R. Kender,et al.  Toward the use of gesture in traditional user interfaces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[10]  Jakub Segen,et al.  Human-computer interaction using gesture recognition and 3D hand tracking , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[11]  Thomas O. Binford,et al.  Interpretation Of Geometric Structure From Image Boundaries , 1981, Other Conferences.

[12]  John R. Kender,et al.  Shape from Darkness: Deriving Surface Information from Dynamic Shadows , 1986, AAAI.