High Resolution Surface Reconstruction from Multi-view Aerial Imagery

This paper presents a novel framework for surface reconstruction from multi-view aerial imagery of large scale urban scenes, which combines probabilistic volumetric modeling with smooth signed distance surface estimation, to produce very detailed and accurate surfaces. Using a continuous probabilistic volumetric model which allows for explicit representation of ambiguities caused by moving objects, reflective surfaces, areas of constant appearance, and self-occlusions, the algorithm learns the geometry and appearance of a scene from a calibrated image sequence. An online implementation of Bayesian learning precess in GPUs significantly reduces the time required to process a large number of images. The probabilistic volumetric model of occupancy is subsequently used to estimate a smooth approximation of the signed distance function to the surface. This step, which reduces to the solution of a sparse linear system, is very efficient and scalable to large data sets. The proposed algorithm is shown to produce high quality surfaces in challenging aerial scenes where previous methods make large errors in surface localization. The general applicability of the algorithm beyond aerial imagery is confirmed against the Middlebury benchmark.

[1]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[2]  Pau Gargallo,et al.  An Occupancy-Depth Generative Model of Multi-view Images , 2007, ACCV.

[3]  Joseph L. Mundy,et al.  Change Detection in a 3-d World , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Pascal Fua,et al.  Efficient large-scale multi-view stereo for ultra high-resolution image sets , 2011, Machine Vision and Applications.

[5]  Edmond Boyer,et al.  Fusion of multiview silhouette cues using a space occupancy grid , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Joseph L. Mundy,et al.  Real-time rendering and dynamic updating of 3-d volumetric data , 2011, GPGPU-4.

[8]  Joseph L. Mundy,et al.  Object Recognition in Probabilistic 3-d Volumetric Scenes , 2012, ICPRAM.

[9]  Roberto Cipolla,et al.  A surface evolution approach of probabilistic space carving , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[10]  Gabriel Taubin,et al.  A continuous probabilistic scene model for aerial imagery , 2010 .

[11]  Gabriel Taubin,et al.  A Variable-Resolution Probabilistic Three-Dimensional Model for Change Detection , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Victor S. Lempitsky,et al.  Oriented Visibility for Multiview Reconstruction , 2006, ECCV.

[13]  R. Cipolla,et al.  A probabilistic framework for space carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Gabriel Taubin,et al.  1 An End-to-End Framework for Evaluating Surface Reconstruction , 2011 .

[15]  Michael Goesele,et al.  Surface Reconstruction from Multi-resolution Sample Points , 2011, VMV.

[16]  Scott Schaefer,et al.  Dual marching cubes: primal contouring of dual grids , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[17]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[19]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[21]  Adam Baumberg,et al.  Blending Images for Texturing 3D Models , 2002, BMVC.

[22]  Shubao Liu,et al.  A complete statistical inverse ray tracing approach to multi-view stereo , 2011, CVPR 2011.

[23]  Victor S. Lempitsky,et al.  Global Optimization for Shape Fitting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Roberto Cipolla,et al.  Probabilistic visibility for multi-view stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Kiriakos N. Kutulakos,et al.  A Probabilistic Theory of Occupancy and Emptiness , 2002, ECCV.

[26]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Matthew J. Sottile,et al.  Curve and surface reconstruction: algorithms with mathematical analysis by Tamal K. Dey Cambridge University Press , 2010, SIGA.

[28]  Daniel Cremers,et al.  Integration of Multiview Stereo and Silhouettes Via Convex Functionals on Convex Domains , 2008, ECCV.

[29]  Gabriel Taubin,et al.  SSD: Smooth Signed Distance Surface Reconstruction , 2011, Comput. Graph. Forum.

[30]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[31]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..