Towards Complete Free-Form Reconstruction of Complex 3D Scenes from an Unordered Set of Uncalibrated Images

This paper describes a method for accurate dense reconstruction of a complex scene from a small set of high-resolution unorganized still images taken by a hand-held digital camera. A fully automatic data processing pipeline is proposed. Highly discriminative features are first detected in all images. Correspondences are then found in all image pairs by wide-baseline stereo matching and used in a scene structure and camera reconstruction step that can cope with occlusion and outliers. Image pairs suitable for dense matching are automatically selected, rectified and used in dense binocular matching. The dense point cloud obtained as the union of all pairwise reconstructions is fused by local approximation using oriented geometric primitives. For texturing, every primitive is mapped on the image with the best resolution.

[1]  Andrew Zisserman,et al.  Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?" , 2002, ECCV.

[2]  Olivier D. Faugeras,et al.  Complete Dense Stereovision Using Level Set Methods , 1998, ECCV.

[3]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[4]  Radim Sára.  Accurate Natural Surface Reconstruction from Polynocular Stereo , 2000, Confluence of Computer Vision and Computer Graphics.

[5]  Luc Van Gool,et al.  Dense matching of multiple wide-baseline views , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Andrew Zisserman,et al.  New Techniques for Automated Architectural Reconstruction from Photographs , 2002, ECCV.

[7]  Long Quan,et al.  Quasi-Dense Reconstruction from Image Sequence , 2002, ECCV.

[8]  Radim Sára,et al.  Dense Stereomatching Algorithm Performance for View Prediction and Structure Reconstruction , 2003, SCIA.

[9]  Radim Sára,et al.  Finding the Largest Unambiguous Component of Stereo Matching , 2002, ECCV.

[10]  Bernd Neumann,et al.  Computer Vision — ECCV’98 , 1998, Lecture Notes in Computer Science.

[11]  Ruzena Bajcsy,et al.  Fish-scales: representing fuzzy manifolds , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[12]  Dmitrij Csetverikov,et al.  Affine dense matching for wide baseline stereo , 2003 .

[13]  Tomás Pajdla,et al.  Consistent Multi-view Reconstruction from Epipolar Geometries with Outliers , 2003, SCIA.

[14]  Mads Nielsen,et al.  Computer Vision — ECCV 2002 , 2002, Lecture Notes in Computer Science.

[15]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[16]  Amitabh Varshney,et al.  Modeling and Rendering of Points with Local Geometry , 2003, IEEE Trans. Vis. Comput. Graph..

[17]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[18]  R. Sára,et al.  Polynocular Image Set Consistency for Local Model Verification , 2000 .

[19]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[20]  Reinhard Koch,et al.  A simple and efficient rectification method for general motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[22]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[24]  Luc Van Gool,et al.  Wide-baseline multiple-view correspondences , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Joachim Bauer,et al.  Modeling and visualizing the cultural heritage data set of Graz , 2001, VAST '01.

[26]  Roberto Cipolla,et al.  Combining single view recognition and multiple view stereo for architectural scenes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[27]  Leszek Wojnar,et al.  Image Analysis , 1998 .

[28]  Song Han,et al.  Reconstructing free-form surfaces from sparse data , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[29]  Kostas Daniilidis,et al.  Stereo-based environment scanning for immersive telepresence , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Marshall W. Bern,et al.  A new Voronoi-based surface reconstruction algorithm , 1998, SIGGRAPH.