ON-LINE INCREMENTAL 3D HUMAN BODY RECONSTRUCTION FOR HMI OR AR APPLICATIONS

This research proposes an on-line incremental 3D reconstruction framework that can be used on human machine interaction (HMI) or augmented reality (AR) applications. There is a wide variety of research opportunities including high performance imaging, multi-view video, virtual view synthesis, etc. One fundamental challenge in geometry reconstruction from traditional cameras array is the lack of accuracy in low-texture or repeated pattern region. Our approach explores virtual view synthesis through motion body estimation and hybrid sensors composed by video cameras and a depth camera based on structured-light or time-of-flight. We present a full 3D body reconstruction system that combines visual features and shape-based alignment. The proposed mesh generation algorithm is based on Crust and efficiently adds new vertices to an already existing surface. Modeling is based on meshes computed from dense depth maps in order lower the data to be processed and create a 3D mesh representation that is independent of view-point.

[1]  Oliver Schreer,et al.  Three-dimensional image processing in the future of immersive media , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Ruzena Bajcsy,et al.  A Framework for Collaborative Real-Time 3D Teleimmersion in a Geographically Distributed Environment , 2008, 2008 Tenth IEEE International Symposium on Multimedia.

[3]  Jan-Michael Frahm,et al.  Towards Urban 3D Reconstruction from Video , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[4]  M. Mattavelli,et al.  Introduction to the special issue on multimedia implementation », IEEE Trans. On Circuits and Systems for Video Technology , 2004 .

[5]  Bruno Raffin,et al.  Grimage: 3D modeling for remote collaboration and telepresence , 2008, VRST '08.

[6]  Jorge Dias,et al.  3D Map Registration using Vision/Laser and Inertial Sensing , 2007, EMCR.

[7]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[8]  João Alves,et al.  Registration and segmentation for 3D map building - a solution based on stereo vision and inertial sensors , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[9]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[11]  Marshall W. Bern,et al.  A new Voronoi-based surface reconstruction algorithm , 1998, SIGGRAPH.

[12]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[13]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.