Using depth measuring cameras for a new human computer interaction in augmented virtual reality environments

The usage of a novel real-time depth-mapping principle, and of a 3D camera which embodies the new depth-mapping principle to control a number of computer applications ranging from games to collaborative multimedia environments, is described in this paper. The 3D camera has a variable depth resolution obtained from images of 1024×1024 pixels. By using the depth data provided by the 3D camera, a person's body parts and their movements are analyzed and reconstructed in real-time. Their features and spatial positions are determined and corresponding actions are triggered. Triggered actions are used to control computer games, digital signage, GIS applications, unmanned vehicles, and consumer electronics such as TVs, set-top boxes and PDAs. In this paper, the use of a 3D camera in a new human computer interface for augmented virtual reality is given and illustrated in a series of images captured from live experiments.

[1]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[2]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[3]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[4]  Patrick J. Flynn,et al.  An evaluation of multimodal 2D+3D face biometrics , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  T. Yorozu,et al.  Electron Spectroscopy Studies on Magneto-Optical Media and Plastic Substrate Interface , 1987, IEEE Translation Journal on Magnetics in Japan.

[6]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[7]  David A. Green,et al.  MDSP: a modular DSP architecture for a real-time 3D laser range sensor , 2002, IS&T/SPIE Electronic Imaging.

[8]  Ying Wu,et al.  Hand modeling, analysis and recognition , 2001, IEEE Signal Process. Mag..

[9]  Michael Lehmann,et al.  An all-solid-state optical range camera for 3D real-time imaging with sub-centimeter depth resolution (SwissRanger) , 2004, SPIE Optical Systems Design.

[10]  Peter M. Will,et al.  Grid Coding: A Preprocessing Technique for Robot and Machine Vision , 1971, IJCAI.

[11]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[12]  Jake K. Aggarwal,et al.  Tracking human motion using multiple cameras , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[13]  Hongming Zhang,et al.  R&D of a Parameterized Method for 3D Virtual Human Body Based on Anthropometry , 2008, Int. J. Virtual Real..

[14]  Nadia Magnenat-Thalmann,et al.  Synthesizing animatable body models with parameterized shape modifications , 2003, SCA '03.

[15]  Ying Wu,et al.  For Vision-Based Human Computer Interaction , 2001 .

[16]  Y. Yoshikuni,et al.  Three-dimensional laser microvision. , 2001, Applied optics.

[17]  Takeshi Naemura,et al.  Acquisition of 3D image representation in multimedia ambiance communication using 3D laser scanner and digital camera , 2000, Electronic Imaging.

[18]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[19]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Heinrich Höfler,et al.  3D profiling by optical demodulation with an image intensifier , 1999, Electronic Imaging.

[21]  Zoran Popovic,et al.  Articulated body deformation from range scan data , 2002, SIGGRAPH.

[22]  Vincent H. Chan,et al.  Spherical/cylindrical laser scanner for geometric reverse engineering , 2004, IS&T/SPIE Electronic Imaging.