Real-Time 4D Reconstruction of Human Motion

A studio for real-time 4D reconstruction of moving actors and deformable objects has been recently created at the Computer and Automation Research Institute of the Hungarian Academy of Sciences (MTA SZTAKI). The studio uses 13 synchronised, calibrated high-resolution video cameras and a GPU to build dynamic 3D models providing free-viewpoint video in real-time. We give a brief overview of advanced studios operating around the world, then discuss the GPU implementation details of the reconstruction pipeline. Finally, a performance comparison of the offline and real-time versions of the system is given.

[1]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Jean-Philippe Pons,et al.  Spatio-temporal image-based texture atlases for dynamic 3-D models , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[3]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[5]  Zsolt Jankó,et al.  4D Reconstruction Studio: Creating dynamic 3D models of moving actors , 2012 .

[6]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[8]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Antonis A. Argyros,et al.  From Multiple Views to Textured 3D Meshes: A GPU-Powered Approach , 2010, ECCV Workshops.