Tackling Shapes and BRDFs Head-On

In this work, we investigate the use of simple flash-based photography to capture an object's 3D shape and reflectance characteristics at the same time. The presented method is based on the principles of Structure from Motion (SfM) and Photometric Stereo (PS), yet, we make sure not to use more than readily-available consumer equipment, like a camera with flash. Starting from a SfM-generated mesh, we apply PS to refine both geometry and reflectance, where the latter is expressed in terms of data-driven Bidirectional Reflectance Distribution Function (BRDF) representations. We also introduce a novel approach to infer complete BRDFs starting from the sparsely sampled data-driven reflectance information captured with this setup. Our approach is experimentally validated by modeling several challenging objects, both synthetic and real.

[1]  Katsushi Ikeuchi,et al.  Elevation Angle from Reflectance Monotonicity: Photometric Stereo for General Isotropic Reflectances , 2012, ECCV.

[2]  Baining Guo,et al.  Pocket reflectometry , 2011, SIGGRAPH 2011.

[3]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[4]  Ko Nishino,et al.  Shape and Reflectance from Natural Illumination , 2012, ECCV.

[5]  Luc Van Gool,et al.  A Public System for Image Based 3D Model Generation , 2011, MIRAGE.

[6]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[7]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[8]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[9]  Todd E. Zickler,et al.  Passive Reflectometry , 2008, ECCV.

[10]  Szymon Rusinkiewicz,et al.  Spacetime Stereo: A Unifying Framework for Depth from Triangulation , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Mashhuda Glencross,et al.  High-resolution relightable buildings from photographs , 2011, SIGGRAPH '11.

[12]  Katsushi Ikeuchi,et al.  Consensus photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Jason Lawrence,et al.  A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance , 2010, ACM Transactions on Graphics.

[14]  Hans-Peter Seidel,et al.  Mesostructure from Specularity , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Ravi Ramamoorthi,et al.  A theory of differential photometric stereo for unknown isotropic BRDFs , 2011, CVPR 2011.

[16]  Lizhuang Ma,et al.  An extended photometric stereo algorithm for recovering specular object shape and its reflectance properties , 2010, Comput. Sci. Inf. Syst..

[17]  Yasuyuki Matsushita,et al.  A hand-held photometric stereo camera for 3-D modeling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Douglas Lanman,et al.  Multi-flash 3D photography: capturing shape and appearance , 2006, SIGGRAPH '06.

[21]  Yasuyuki Matsushita,et al.  Multiview Photometric Stereo Using Planar Mesh Parameterization , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Ramesh Raskar,et al.  Removing photography artifacts using gradient projection and flash-exposure sampling , 2005, SIGGRAPH 2005.

[23]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[24]  Hans-Peter Kriegel,et al.  Subspace clustering , 2012, WIREs Data Mining Knowl. Discov..

[25]  Zhe Wu,et al.  Multi-view Photometric Stereo with Spatially Varying Isotropic Materials , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  R. Cipolla,et al.  Multi-view photometric stereo , 2007 .

[27]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[28]  Jason Lawrence,et al.  A photometric approach for estimating normals and tangents , 2008, ACM Trans. Graph..

[29]  Pieter Peers,et al.  Estimating Specular Roughness and Anisotropy from Second Order Spherical Gradient Illumination , 2009, Comput. Graph. Forum.

[30]  Ko Nishino,et al.  Single image multimaterial estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[32]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Luc Van Gool,et al.  Base Materials for Photometric Stereo , 2012, ECCV Workshops.

[34]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Daniel G. Aliaga,et al.  Photogeometric structured light: A self-calibrating and multi-viewpoint framework for accurate 3D modeling , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Long Quan,et al.  The Geometry of Reflectance Symmetries , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  John M. Snyder,et al.  Manifold bootstrapping for SVBRDF capture , 2010, ACM Trans. Graph..

[38]  Steven M. Seitz,et al.  Shape and Spatially-Varying BRDFs from Photometric Stereo , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Christopher Schwartz,et al.  Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage , 2011, VAST.

[40]  Ravi Ramamoorthi,et al.  What an image reveals about material reflectance , 2011, 2011 International Conference on Computer Vision.

[41]  Li Zhang,et al.  Shape and motion under varying illumination , 2003, IEEE International Conference on Computer Vision.