Depth video based human model reconstruction resolving self-occlusion

Generation of a 3D model has been actively researched in computer vision area. Especially, modeling of a realistic human is a topic of great interest due to its use in many application areas such as animation and 3DTV. In this paper, a novel method is presented to generate a 3D human model using a sequence of depth images from a monocular TOF (time-of-flight) camera. Mainly, we solve a 'self-occlusion' problem which occurs in the depth data where a human model is partially occluded by other parts of a human body. This becomes very critical when we want to develop a human model for various applications. We demonstrate the capability of our method by showing experimental results that resolves a multiple occlusion problem.

[1]  Markus H. Gross,et al.  Point-sampled 3D video of real-world scenes , 2007, Signal Process. Image Commun..

[2]  Borut Zalik,et al.  An efficient chain code with Huffman coding , 2005, Pattern Recognit..

[3]  Warnakulasuriya Anil Chandana Fernando,et al.  Object based coding of the depth maps for 3D video coding , 2009, IEEE Transactions on Consumer Electronics.

[4]  Behzad Dariush,et al.  Controlled human pose estimation from depth image streams , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[5]  Jongeun Cha,et al.  Depth Video Enhancement for Haptic Interaction Using a Smooth Surface Reconstruction , 2006, IEICE Trans. Inf. Syst..

[6]  Yo-Sung Ho,et al.  Dynamic 3D human actor generation method using a time-of-flight depth camera , 2008, IEEE Transactions on Consumer Electronics.

[7]  G. Iddan,et al.  3D IMAGING IN THE STUDIO (AND ELSEWHERE...) , 2001 .

[8]  Vid Domiter.  Constrained Delaunay Triangulation using Plane Subdivision , 2004 .

[9]  Stefan May,et al.  Calibration and Registration for Precise Surface Reconstruction with TOF Cameras , 2007 .

[10]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[11]  Bob Stein.  A Point About Polygons , 1997 .

[12]  Guilherme N. DeSouza,et al.  3-D Modeling of Real-World Objects Using Range and Intensity Images , 2005, Machine Learning and Robot Perception.

[13]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[14]  Peter Liepa,et al.  Filling Holes in Meshes , 2003, Symposium on Geometry Processing.

[15]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[16]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[17]  Marc Pollefeys,et al.  Visual Hull Construction in the Presence of Partial Occlusion , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[18]  Song Zhang,et al.  High-resolution, real-time three-dimensional shape measurement , 2006 .