View-dependent 3 D Projection using Depth-Image-based Head Tracking

The idea of augmenting our physical reality with virtual entities in an unobtrusive, embedded way, i. e., without the necessity of wearing hardware or using expensive display devices is exciting. Due to the development of new inexpensive depth camera systems an implementation of this idea became affordable for everyone. This paper demonstrates an approach to capture and track a head using a depth camera. From its position in space we will calculate a viewdependent 3D projection of a scene which can be projected on walls, on tables, or on any other type of flat surface. Our projector-camera system enables the viewer to explore the projected scene while walking around.

[1]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[2]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[3]  Alexander Zelinsky,et al.  Real-time stereo tracking for head pose and gaze estimation , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[4]  Martin Herman,et al.  Head tracking using stereo , 2002, Machine Vision and Applications.

[5]  Carlo Tomasi,et al.  3D head tracking based on recognition and interpolation using a time-of-flight depth sensor , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[6]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[7]  Q. M. Jonathan Wu,et al.  Real-Time 3D Head Tracking Based on Time-of-Flight Depth Sensor , 2007, 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007).

[8]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[9]  Johnny Chung Lee,et al.  Hacking the Nintendo Wii Remote , 2008, IEEE Pervasive Computing.

[10]  Jiaqing Lin,et al.  Free hand interface for controlling applications based on Wii remote IR sensor , 2010, VRCAI '10.