Better Together: Shading Cues and Multi-View Stereo for Reconstruction Depth Optimization

Passive reconstruction methods such as traditional multi-view stereo are capable to accurate reconstruction results. However, the depth calculation of multi-view stereo encounters significant difficulties, especially when the corresponding points have some degree of inaccuracy or unreliability. In this paper, we make use of geometric and shading cues information in the multi-view stereo approach to construct a robust energy function, which is also effective to optimize the depth information provided by multi-view stereo. This work improves the accuracy of point cloud depth. we evaluated our algorithm in the famous DTU datasets, and established our own dinosaur datasets. All the data is collected by mobile devices under natural light condition, while the reconstruction results are completed and accurate.

[1]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Henrik Aanæs,et al.  Large Scale Multi-view Stereopsis Evaluation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[4]  Olivier D. Faugeras,et al.  Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score , 2007, International Journal of Computer Vision.

[5]  Pascal Monasse,et al.  OpenMVG: Open Multiple View Geometry , 2016, RRPR@ICPR.

[6]  Yasuyuki Matsushita,et al.  High-quality shape from multi-view stereo and shading under general illumination , 2011, CVPR 2011.

[7]  Andrew W. Fitzgibbon,et al.  PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation , 2014, International Journal of Computer Vision.

[8]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Konrad Schindler,et al.  Massively Parallel Multiview Stereopsis by Surface Normal Diffusion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Matthias Nießner,et al.  Shading-based refinement on volumetric signed distance functions , 2015, ACM Trans. Graph..

[11]  Deyu Meng,et al.  Multi-view stereo via depth map fusion: A coordinate decent optimization method , 2016, Neurocomputing.

[12]  Ben Semerjian,et al.  A New Variational Framework for Multiview Surface Reconstruction , 2014, ECCV.

[13]  Jan-Michael Frahm,et al.  Pixelwise View Selection for Unstructured Multi-View Stereo , 2016, ECCV.

[14]  Carsten Rother,et al.  PatchMatch Stereo - Stereo Matching with Slanted Support Windows , 2011, BMVC.

[15]  Michael Breuß,et al.  Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth, Illumination and Albedo , 2018, International Journal of Computer Vision.

[16]  Jean-Philippe Pons,et al.  Efficient Multi-View Reconstruction of Large-Scale Scenes using Interest Points, Delaunay Triangulation and Graph Cuts , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Pascal Fua,et al.  Efficient large-scale multi-view stereo for ultra high-resolution image sets , 2011, Machine Vision and Applications.

[18]  Jan-Michael Frahm,et al.  Real-Time Visibility-Based Fusion of Depth Maps , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Roberto Cipolla,et al.  Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo , 2008, ECCV.

[20]  Long Quan,et al.  A Surface Reconstruction Method Using Global Graph Cut Optimization , 2006, International Journal of Computer Vision.

[21]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[22]  Berthold K. P. Horn.  Height and gradient from shading , 1989, International Journal of Computer Vision.

[23]  Luc Van Gool,et al.  Combined Depth and Outlier Estimation in Multi-View Stereo , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Julian Eggert,et al.  Stereo image warping for improved depth estimation of road surfaces , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[26]  Michael Goesele,et al.  Shading-Aware Multi-view Stereo , 2016, ECCV.

[27]  Daniel Cremers,et al.  3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination , 2008, International Journal of Computer Vision.