From Multiple Views to Textured 3D Meshes: A GPU-Powered Approach

We present work on exploiting modern graphics hardware towards the real-time production of a textured 3D mesh representation of a scene observed by a multicamera system. The employed computational infrastructure consists of a network of four PC workstations each of which is connected to a pair of cameras. One of the PCs is equipped with a GPU that is used for parallel computations. The result of the processing is a list of texture mapped triangles representing the reconstructed surfaces. In contrast to previous works, the entire processing pipeline (foreground segmentation, 3D reconstruction, 3D mesh computation, 3D mesh smoothing and texture mapping) has been implemented on the GPU. Experimental results demonstrate that an accurate, high resolution, texture-mapped 3D reconstruction of a scene observed by eight cameras is achievable in real time.

[1]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[2]  Peter Eisert,et al.  Parallel high resolution real-time Visual Hull on GPU , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[3]  Hong Yi,et al.  A survey of the marching cubes algorithm , 2006, Comput. Graph..

[4]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[5]  Yao Zhang,et al.  Scan primitives for GPU computing , 2007, GH '07.

[6]  Michael Potmesil.  Generating octree models of 3D objects from their silhouettes in a sequence of images , 1987, Comput. Vis. Graph. Image Process..

[7]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[8]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[9]  Itaru Kitahara,et al.  Compensated Visual Hull with GPU-Based Optimization , 2008, PCM.

[10]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[11]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[12]  Bohyung Han,et al.  SEQUENTIAL KERNEL DENSITY APPROXIMATION THROUGH MODE PROPAGATION: APPLICATIONS TO BACKGROUND MODELING , 2004 .

[13]  Hamish A. Carr,et al.  Accelerating marching cubes with graphics hardware , 2006, CASCON.

[14]  Günther Greiner,et al.  Realtime Isosurface Extraction with Graphics Hardware , 2004, Eurographics.

[15]  V. Pascucci,et al.  Isosurface computation made simple: hardware acceleration, adaptive refinement and tetrahedral stripping , 2004, VISSYM'04.

[16]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[17]  Rainer Stiefelhagen,et al.  Real-Time GPU-Based Voxel Carving with Systematic Occlusion Handling , 2009, DAGM-Symposium.

[18]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[19]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[20]  Nassir Navab,et al.  Efficient visual hull computation for real-time 3D reconstruction using CUDA , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[21]  Mohan M. Trivedi,et al.  Detecting Moving Shadows: Algorithms and Evaluation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Simon Stegmaier,et al.  Hardware-accelerated reconstruction of polygonal isosurface representations on unstructured grids , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[24]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[25]  Susan Hackwood,et al.  Computational geometric methods in volumetric intersection for 3D reconstruction , 1990, Pattern Recognit..

[26]  Antonis A. Argyros,et al.  A checkerboard detection utility for intrinsic and extrinsic camera cluster calibration , 2009 .

[27]  Thomas Malzbender,et al.  A Survey of Methods for Volumetric Scene Reconstruction from Photographs , 2001, VG.

[28]  Changsheng Xu,et al.  Advances in Multimedia Information Processing - PCM 2008, 9th Pacific Rim Conference on Multimedia, Tainan, Taiwan, December 9-13, 2008. Proceedings , 2008, PCM.

[29]  Gitta Domik,et al.  Real-Time Marching Cubes on the Vertex Shader , 2005, Eurographics.