Multi-Resolution Range Data Fusion for Multi-View Stereo Reconstruction

In this paper we present a probabilistic algorithm for multi-view reconstruction from calibrated images. The algorithm is based on multi-resolution volumetric range image integration and is highly separable as it only employs local optimization. Dense depth maps are transformed in an octree data structure with variable voxel sizes. This allows for an efficient modeling of point clouds with very variable density. A probability function constructed in discrete space is built locally with a Bayesian approach. Compared to other algorithms we can deal with extremely big scenes and complex camera configurations in a limited amount of time, as the solution can be split in arbitrarily many parts and computed in parallel. The algorithm has been applied to lab and outdoor benchmark data as well as to large image sets of urban regions taken by cameras on Unmanned Aerial Vehicles (UAVs) and from the ground, demonstrating high surface quality and good runtime performance.

[1]  Michael Goesele,et al.  Hierarchical Surface Reconstruction from Multi-resolution Point Samples , 2011, Theoretical Foundations of Computer Vision.

[2]  Heiko Hirschmüller,et al.  Stereo vision based mapping and immediate virtual walkthroughs , 2003 .

[3]  George Vogiatzis,et al.  WITHDRAWN: Video-based, real-time multi-view stereo , 2012 .

[4]  C. Zach Fast and High Quality Fusion of Depth Maps , 2008 .

[5]  Jan-Michael Frahm,et al.  3D Reconstruction Using an n-Layer Heightmap , 2010, DAGM-Symposium.

[6]  Tim Bodenmüller,et al.  Streaming surface reconstruction from real time 3D-measurements , 2009 .

[7]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Carlos Hernández,et al.  Video-based, real-time multi-view stereo , 2011, Image Vis. Comput..

[9]  George Vogiatzis,et al.  A Generative Model for Online Depth Fusion , 2012, ECCV.

[10]  Luc Van Gool,et al.  Combined Depth and Outlier Estimation in Multi-View Stereo , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Tomás Pajdla,et al.  Scalable multi-view stereo , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[12]  M. Goesele,et al.  Fusion of depth maps with multiple scales , 2011, ACM Trans. Graph..

[13]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Horst Bischof,et al.  A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Matthieu Guillaumin,et al.  Segmentation Propagation in ImageNet , 2012, ECCV.

[16]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[17]  Michael Goesele,et al.  Surface Reconstruction from Multi-resolution Sample Points , 2011, VMV.

[18]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[20]  Jan-Michael Frahm,et al.  Real-Time Visibility-Based Fusion of Depth Maps , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  Roberto Cipolla,et al.  Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo , 2008, ECCV.

[22]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Li Guan,et al.  3D Object Reconstruction with Heterogeneous Sensor Data , 2008 .

[25]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Jean-Philippe Pons,et al.  High Accuracy and Visibility-Consistent Dense Multiview Stereo , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).