Fast Omnidirectional 3D Scene Acquisition with an Array of Stereo Cameras

We present an omnidirectional 3D acquisition system based on a mobile array of high-resolution consumer digital SLR cameras that automatically capture high dynamic range stereo pairs across a full 360-degree panorama. The stereo pairs are augmented with a time-varying lighting pattern created using standard photographic flashes, lenses, and patterned slides. Spacetime stereo techniques are used to generate 3D range images with corresponding color data from the HDR photographs. The multiple range images are aligned with egomotion estimation and ICP registration techniques, and volumetric merging and color texturing algorithms allow the rapid creation of complete 3D models. The resulting system compares favorably with other state of the art 3D acquisition technologies in the resolution and quality of its output, and can be faster and less expensive than 3D laser scanners for digitizing large 3D scenes such as building interiors.

[1]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Linda G. Shapiro,et al.  Surface Reconstruction and Display from Range and Color Data , 2000, Graph. Model..

[3]  Wolfgang Straßer,et al.  Omnidirectional 3D Modeling on a Mobile Robot using Graph Cuts , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[4]  Rui Wang,et al.  Efficient reconstruction of indoor scenes with color , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[5]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[6]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[7]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[8]  Steve Marschner,et al.  Filling holes in complex surfaces using volumetric diffusion , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[9]  Piotr Jasiobedzki,et al.  Photo-realistic 3D model reconstruction , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[10]  Szymon Rusinkiewicz,et al.  Improved sub-pixel stereo correspondences through symmetric refinement , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Andrew Zisserman,et al.  Multiple View Geometry , 1999 .

[12]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[13]  Kim L. Boyer,et al.  Color-Encoded Structured Light for Rapid Active Ranging , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[16]  Sunil Arya,et al.  ANN: library for approximate nearest neighbor searching , 1998 .

[17]  Voicu Popescu,et al.  The ModelCamera: a hand-held device for interactive modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[18]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  Anselmo Lastra,et al.  Automatic image alignment for 3D environment modeling , 2004, Proceedings. 17th Brazilian Symposium on Computer Graphics and Image Processing.

[20]  Sebastian Weik,et al.  Registration of 3-D partial surface models using luminance and depth information , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[21]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[22]  Ting Li,et al.  A robust image-based method for 3D registration , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[23]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[24]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Szymon Rusinkiewicz,et al.  Spacetime stereo: a unifying framework for depth from triangulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Matthew A. Brown,et al.  Unsupervised 3D object recognition and reconstruction in unordered datasets , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[27]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[28]  Ben J. A. Kröse,et al.  Robust scene reconstruction from an omnidirectional vision system , 2003, IEEE Trans. Robotics Autom..

[29]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[30]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[32]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.