Shading‐Based Surface Recovery Using Subdivision‐Based Representation

This paper presents subdivision‐based representations for both lighting and geometry in shape‐from‐shading. A very recent shading‐based method introduced a per‐vertex overall illumination model for surface reconstruction, which has advantage of conveniently handling complicated lighting condition and avoiding explicit estimation of visibility and varied albedo. However, due to its discrete nature, the per‐vertex overall illumination requires a large amount of memory and lacks intrinsic coherence. To overcome these problems, in this paper we propose to use classic subdivision to define the basic smooth lighting function and surface, and introduce additional independent variables into the subdivision to adaptively model sharp changes of illumination and geometry. Compared to previous works, the new model not only preserves the merits of the per‐vertex illumination model, but also greatly reduces the number of variables required in surface recovery and intrinsically regularizes the illumination vectors and the surface. These features make the new model very suitable for multi‐view stereo surface reconstruction under general, unknown illumination condition. Particularly, a variational surface reconstruction method built upon the subdivision representations for lighting and geometry is developed. The experiments on both synthetic and real‐world data sets have demonstrated that the proposed method can achieve memory efficiency and improve surface detail recovery.

[1]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Georg Umlauf,et al.  Analysis and tuning of subdivision algorithms , 2005, SCCG '05.

[3]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.

[4]  Michael Wimmer,et al.  Efficient irradiance normal mapping , 2010, I3D '10.

[5]  Jitendra Malik,et al.  Intrinsic Scene Properties from a Single RGB-D Image , 2013, CVPR.

[6]  M. Goesele,et al.  Floating scale surface reconstruction , 2014, ACM Trans. Graph..

[7]  Zheng Li,et al.  Deducing interpolating subdivision schemes from approximating subdivision schemes , 2008, SIGGRAPH Asia '08.

[8]  Masatoshi Okutomi,et al.  Multi-view Inverse Rendering Under Arbitrary Illumination and Albedo , 2016, ECCV.

[9]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[11]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[12]  Jitendra Malik,et al.  Intrinsic Scene Properties from a Single RGB-D Image , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  James T. Kajiya,et al.  The rendering equation , 1986, SIGGRAPH.

[15]  Jianfei Cai,et al.  Shading-Based Surface Detail Recovery Under General Unknown Illumination , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Radu Horaud,et al.  TransforMesh : A Topology-Adaptive Mesh-Based Approach to Surface Evolution , 2007, ACCV.

[17]  Anders Bjorholm Dahl,et al.  Large-Scale Data for Multiple-View Stereopsis , 2016, International Journal of Computer Vision.

[18]  Peng Song,et al.  Volumetric stereo and silhouette fusion for image-based modeling , 2010, The Visual Computer.

[19]  Charles T. Loop,et al.  Smooth Subdivision Surfaces Based on Triangles , 1987 .

[20]  Jan-Michael Frahm,et al.  Real-Time Visibility-Based Fusion of Depth Maps , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  Radim Sara,et al.  Refinement of Surface Mesh for Accurate Multi-View Reconstruction , 2010, Int. J. Virtual Real..

[22]  Sumanta N. Pattanaik,et al.  Eurographics Symposium on Rendering (2004) a Novel Hemispherical Basis for Accurate and Efficient Rendering , 2022 .

[23]  Hans-Peter Seidel,et al.  Shading-based dynamic shape refinement from multi-view video under general illumination , 2011, 2011 International Conference on Computer Vision.

[24]  Ko Nishino,et al.  Multiview Shape and Reflectance from Natural Illumination , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Henning Biermann,et al.  Sharp Features on Multiresolution Subdivision Surfaces , 2002, Graph. Model..

[26]  Roberto Cipolla,et al.  Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo , 2008, ECCV.

[27]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Marc Pollefeys,et al.  Multi-View Stereo via Graph Cuts on the Dual of an Adaptive Tetrahedral Mesh , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[29]  Tomás Pajdla,et al.  Multi-view reconstruction preserving weakly-supported surfaces , 2011, CVPR 2011.

[30]  Michael Goesele,et al.  Shading-Aware Multi-view Stereo , 2016, ECCV.

[31]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[32]  David A. Forsyth,et al.  Variable-Source Shading Analysis , 2011, International Journal of Computer Vision.

[33]  Robin Green,et al.  Spherical Harmonic Lighting: The Gritty Details , 2003 .

[34]  Matthias Nießner,et al.  Shading-based refinement on volumetric signed distance functions , 2015, ACM Trans. Graph..

[35]  Leif Kobbelt,et al.  A Surface-Growing Approach to Multi-View Stereo Reconstruction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.