Approximate svBRDF Estimation From Mobile Phone Video

We describe a new technique for obtaining a spatially varying BRDF (svBRDF) of a flat object using printed fiducial markers and a cell phone capable of continuous flash video. Our homography-based video frame alignment method does not require the fiducial markers to be visible in every frame, thereby enabling us to capture larger areas at a closer distance and higher resolution than in previous work. Pixels in the resulting panorama are fit with a BRDF based on a recursive subdivision algorithm, utilizing all the light and view positions obtained from the video. We show the versatility of our method by capturing a variety of materials with both one and two camera input streams and rendering our results on 3D objects under complex illumination.

[1]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[2]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[3]  Paul Graham,et al.  Acquiring reflectance and shape from continuous spherical harmonic illumination , 2013, ACM Trans. Graph..

[4]  Ravi Ramamoorthi,et al.  Reflectance sharing: predicting appearance from a sparse set of images of a known shape , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Zhe Wu,et al.  Multi-view Photometric Stereo with Spatially Varying Isotropic Materials , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[7]  E. Malis,et al.  Deeper understanding of the homography decomposition for vision-based control , 2007 .

[8]  Pieter Peers,et al.  genBRDF: discovering new analytic BRDFs with genetic programming , 2014, ACM Trans. Graph..

[9]  Philippe Bekaert,et al.  Gloss and Normal Map Acquisition of Mesostructures Using Gray Codes , 2009, ISVC.

[10]  Jaakko Lehtinen,et al.  Reflectance modeling by neural texture synthesis , 2016, ACM Trans. Graph..

[11]  Baining Guo,et al.  Image‐based Material Weathering , 2008, Comput. Graph. Forum.

[12]  John M. Snyder,et al.  Manifold bootstrapping for SVBRDF capture , 2010, ACM Trans. Graph..

[13]  F. E. Nicodemus.  Directional Reflectance and Emissivity of an Opaque Surface , 1965 .

[14]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[15]  Hans-Peter Seidel,et al.  Image-based reconstruction of spatial appearance and geometric detail , 2003, TOGS.

[16]  Anselmo Lastra,et al.  Efficient rendering of spatial bi-directional reflectance distribution functions , 2002, HWWS '02.

[17]  Fabio Pellacini,et al.  AppIm , 2014, ACM Trans. Graph..

[18]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[19]  Jaakko Lehtinen,et al.  Practical SVBRDF capture in the frequency domain , 2013, ACM Trans. Graph..

[20]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Xiao Li,et al.  Modeling surface appearance from a single photograph using self-augmented convolutional neural networks , 2017, ACM Trans. Graph..

[22]  Yong Yu,et al.  Sparse-as-possible SVBRDF acquisition , 2016, ACM Trans. Graph..

[23]  F. E. Nicodemus,et al.  Geometrical considerations and nomenclature for reflectance , 1977 .

[24]  Sabine Süsstrunk,et al.  Handheld Reflectance Acquisition of Paintings , 2017, IEEE Transactions on Computational Imaging.

[25]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Abhijeet Ghosh,et al.  Polarization imaging reflectometry in the wild , 2017, ACM Trans. Graph..

[27]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[28]  Jian Wang,et al.  Reflectance Capture Using Univariate Sampling of BRDFs , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Abhijeet Ghosh,et al.  Circularly polarized spherical illumination reflectometry , 2010, SIGGRAPH 2010.

[30]  Eustace L. Dereniak,et al.  The Theory And Measurement Of Bidirectional Reflectance Distribution Function (Brdf) And Bidirectional Transmittance Distribution Function (BTDF) , 1981, Other Conferences.

[31]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[32]  Jaakko Lehtinen,et al.  Two-shot SVBRDF capture for stationary materials , 2015, ACM Trans. Graph..

[33]  John M. Snyder,et al.  Modeling anisotropic surface reflectance with example-based microfacet synthesis , 2008, SIGGRAPH 2008.

[34]  Pieter Peers,et al.  Estimating specular roughness from polarized second order spherical gradient illumination , 2009, SIGGRAPH '09.

[35]  Jannik Boll Nielsen,et al.  Minimal BRDF sampling for two-shot near-field reflectance acquisition , 2016, ACM Trans. Graph..

[36]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[37]  Baining Guo,et al.  AppGen: interactive material modeling from a single image , 2011, ACM Trans. Graph..

[38]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[39]  Aswin C. Sankaranarayanan,et al.  Shape and Spatially-Varying Reflectance Estimation from Virtual Exemplars , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Pieter Peers,et al.  Mobile Surface Reflectometry , 2016, Comput. Graph. Forum.

[41]  Niklas Bergström,et al.  Detecting, segmenting and tracking unknown objects using multi-label MRF inference , 2014, Comput. Vis. Image Underst..

[42]  Steven M. Seitz,et al.  Shape and Spatially-Varying BRDFs from Photometric Stereo , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .