Dense 3D Point Cloud Generation from Multiple High-resolution Spherical Images

The generation of virtual models of cultural heritage assets is of high interest for documentation, restoration, development and promotion purposes. To this aim, non-invasive, easy and automatic techniques are required. We present a technology that automatically reconstructs large scale scenes from panoramic, high-resolution, spherical images. The advantage of the spherical panoramas is that they can acquire a complete environment in one single image. We show that the spherical geometry is more suited for the computation of the orientation of the panoramas (Structure from Motion) than the standard images, and introduce a generic error function for the epipolar geometry of spherical images. We then show how to produce a dense representation of the scene with up to 100 million points, that can serve as input for meshing and texturing software or for computer aided reconstruction. We demonstrate the applicability of our concept with reconstruction of complex scenes in the scope of cultural heritage documentation at the Chinese National Palace Museum of the Forbidden City in Beijing.

[1]  Adrian Hilton,et al.  3D Reconstruction Using Spherical Images , 2006 .

[2]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Didier Stricker,et al.  Robust Point Matching in HDRI through Estimation of Illumination Distribution , 2011, DAGM-Symposium.

[5]  Andrew Zisserman,et al.  Multiple View Geometry , 1999 .

[6]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[7]  Adrian Hilton,et al.  3D Modelling of Static Environments Using Multiple Spherical Stereo , 2010, ECCV Workshops.

[8]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[9]  Reinhard Koch,et al.  An Analysis-by-Synthesis Camera Tracking Approach Based on Free-Form Surfaces , 2007, DAGM-Symposium.

[10]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[11]  H. Bischof,et al.  Region matching for omnidirectional images using virtual camera planes , 2006 .

[12]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Reinhard Koch,et al.  Lens Model Selection for Visual Tracking , 2005, DAGM-Symposium.

[14]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[15]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[16]  Shigang Li,et al.  Spherical stereo for the construction of immersive VR environment , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[17]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[19]  N. Yokoya,et al.  Omni-directional Multi-baseline Stereo without Similarity Measures , 2005 .

[20]  H. M. Karara,et al.  Direct Linear Transformation from Comparator Coordinates into Object Space Coordinates in Close-Range Photogrammetry , 2015 .

[21]  Tomás Pajdla,et al.  Structure from motion with wide circular field of view cameras , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[23]  Jean-Michel Morel,et al.  ASIFT: A New Framework for Fully Affine Invariant Image Comparison , 2009, SIAM J. Imaging Sci..

[24]  Maarten Vergauwen,et al.  Web-based 3D Reconstruction Service , 2006, Machine Vision and Applications.

[25]  Fabio Remondino,et al.  Automation in Multi-Image Spherical Photogrammetry for 3D Architectural Reconstructions , 2010, VAST.

[26]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[27]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[28]  Didier Stricker,et al.  SIFT in perception-based color space , 2010, 2010 IEEE International Conference on Image Processing.

[29]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.