SparkleVision: Seeing the world through random specular microfacets

In this paper, we study the problem of reproducing the light from a single image of an object covered with random specular microfacets on the surface. We show that such reflectors can be interpreted as a randomized mapping from the lighting to the image. Such specular objects have very different optical properties from both diffuse surfaces and smooth specular objects like metals, so we design a special imaging system to robustly and effectively photograph them. We present simple yet reliable algorithms to calibrate the proposed system and do the inference. We conduct experiments to verify the correctness of our model assumptions and prove the effectiveness of our pipeline.

[1]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[2]  William T. Freeman,et al.  Diffuse reflectance imaging with astronomical applications , 2011, 2011 International Conference on Computer Vision.

[3]  Antonio Torralba,et al.  Accidental pinhole and pinspeck cameras: Revealing the scene outside the picture , 2012, CVPR.

[4]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[5]  2013 Ieee International Conference on Computer Vision , 2022 .

[6]  Marc Levoy,et al.  Light Fields and Computational Imaging , 2006, Computer.

[7]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Shree K. Nayar,et al.  Eyes for relighting , 2004, ACM Trans. Graph..

[9]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[10]  Ohad Ben-Shahar,et al.  A linear formulation of shape from specular flow , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[12]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[13]  Ohad Ben-Shahar,et al.  Shape from specular flow: Is one flow enough? , 2011, CVPR 2011.

[14]  Ohad Ben-Shahar,et al.  Toward a Theory of Shape from Specular Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  John Wright,et al.  Toward Guaranteed Illumination Models for Non-convex Objects , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Pat Hanrahan,et al.  A signal-processing framework for reflection , 2004, ACM Trans. Graph..

[17]  R. Fergus,et al.  Random Lens Imaging , 2006 .

[18]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[19]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[20]  H. Bülthoff,et al.  Does the brain know the physics of specular reflection? , 1990, Nature.

[21]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[22]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[23]  Antonio Torralba,et al.  Accidental Pinhole and Pinspeck Cameras , 2014, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..