Robust and Efficient Photo-Consistency Estimation for Volumetric 3D Reconstruction

Estimating photo-consistency is one of the most important ingredients for any 3D stereo reconstruction technique that is based on a volumetric scene representation. This paper presents a new, illumination invariant photo-consistency measure for high quality, volumetric 3D reconstruction from calibrated images. In contrast to current standard methods such as normalized cross-correlation it supports unconstrained camera setups and non-planar surface approximations. We show how this measure can be embedded into a highly efficient, completely hardware accelerated volumetric reconstruction pipeline by exploiting current graphics processors. We provide examples of high quality reconstructions with computation times of only a few seconds to minutes, even for large numbers of cameras and high volumetric resolutions.

[1]  Matthias Zwicker,et al.  High-quality surface splatting on today's GPUs , 2005, Proceedings Eurographics/IEEE VGTC Symposium Point-Based Graphics, 2005..

[2]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[3]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[4]  R. Sára,et al.  Polynocular Image Set Consistency for Local Model Verification , 2000 .

[5]  Hans-Peter Seidel,et al.  Hardware‐Accelerated Rendering of Photo Hulls , 2004, Comput. Graph. Forum.

[6]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Ronald W. Schafer,et al.  Image-based photo hulls for fast and photo-realistic new view synthesis , 2003, Real Time Imaging.

[8]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[9]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[10]  Ruigang Yang,et al.  Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Thomas Malzbender,et al.  Generalized Voxel Coloring , 1999, Workshop on Vision Algorithms.

[12]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[13]  Mark R. Stevens,et al.  A histogram-based color consistency test for voxel coloring , 2002, Object recognition supported by user interaction for service robots.

[14]  Carlos Hernández Esteban,et al.  Modélisation d'objets 3D par fusion silhouettes-stéréo à partir de séquences d'images en rotation non calibrées. (Stereo and silhouette fusion for 3D object modeling from uncalibrated images under circular motion) , 2004 .

[15]  Nader Bagherzadeh,et al.  Hardware Accelerated Voxel Carving , 2002 .

[16]  Paul A. Viola,et al.  Roxels: responsibility weighted 3D volume reconstruction , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[18]  Peter Eisert,et al.  Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[19]  Roberto Cipolla,et al.  A Probabilistic Framework for Space Carving , 2001, ICCV.