SparkleVision : Seeing the world through random specular

In this paper, we study the problem of reproducing the world lighting from a single image of an object covered with random specular microfacets on the surface. We show that such reflectors can be interpreted as a randomized mapping from the lighting to the image. Such specular objects have very different optical properties from both diffuse surfaces and smooth specular objects like metals, so we design special imaging system to robustly and effectively photograph them. We present simple yet reliable algorithms to calibrate the proposed system and do the inference. We conduct experiments to verify the correctness of our model assumptions and prove the effectiveness of our pipeline.

[1]  H. Bülthoff,et al.  Does the brain know the physics of specular reflection? , 1990, Nature.

[2]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[4]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Pat Hanrahan,et al.  A signal-processing framework for reflection , 2004, ACM Trans. Graph..

[7]  Shree K. Nayar,et al.  Eyes for relighting , 2004, SIGGRAPH 2004.

[8]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[10]  Marc Levoy,et al.  Light Fields and Computational Imaging , 2006, Computer.

[11]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[12]  R. Fergus,et al.  Random Lens Imaging , 2006 .

[13]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[14]  Ohad Ben-Shahar,et al.  Toward a Theory of Shape from Specular Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Ohad Ben-Shahar,et al.  A linear formulation of shape from specular flow , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Ohad Ben-Shahar,et al.  Shape from specular flow: Is one flow enough? , 2011, CVPR 2011.

[17]  William T. Freeman,et al.  Diffuse reflectance imaging with astronomical applications , 2011, 2011 International Conference on Computer Vision.

[18]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[19]  Antonio Torralba,et al.  Accidental pinhole and pinspeck cameras: Revealing the scene outside the picture , 2012, CVPR.

[20]  John Wright,et al.  Toward Guaranteed Illumination Models for Non-convex Objects , 2013, 2013 IEEE International Conference on Computer Vision.