Augmenting fast stereo with silhouette constraints for dynamic 3D capture

We propose and evaluate a method that combines a visual hull with multi-view stereo for the fast acquisition of dynamic 3D data. Both use the same set of cameras. The focus lies on high speed, as 3D has to be generated repeatedly, based on the output of multiple video cameras. The underlying principle is to bootstrap the system with quickly computed sparse stereo, constrained by silhouette information. We further show how this integrates with a fast plane sweeping algorithm, as our dense stereo algorithm of choice

[1]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Luc Van Gool,et al.  Real-time connectivity constrained depth map computation using programmable graphics hardware , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Hans-Peter Seidel,et al.  Combining stereo and visual hull information for on-line reconstruction and rendering of dynamic scenes , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[4]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[5]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[6]  Jean Ponce,et al.  On computing exact visual hulls of solids bounded by smooth surfaces , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Luc Van Gool,et al.  GPU-Based Foreground-Background Segmentation using an Extended Colinearity Criterion , 2005 .

[8]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[9]  Takeo Kanade,et al.  Visual hull alignment and refinement across time: a 3D reconstruction algorithm combining shape-from-silhouette with stereo , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[11]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[12]  Ronald W. Schafer,et al.  Image-based photo hulls for fast and photo-realistic new view synthesis , 2003, Real Time Imaging.

[13]  Marc Pollefeys,et al.  Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.