Shape, albedo, and illumination from a single image of an unknown object

We address the problem of recovering shape, albedo, and illumination from a single grayscale image of an object, using shading as our primary cue. Because this problem is fundamentally underconstrained, we construct statistical models of albedo and shape, and define an optimization problem that searches for the most likely explanation of a single image. We present two priors on albedo which encourage local smoothness and global sparsity, and three priors on shape which encourage flatness, outward-facing orientation at the occluding contour, and local smoothness. We present an optimization technique for using these priors to recover shape, albedo, and a spherical harmonic model of illumination. Our model, which we call SAIFS (shape, albedo, and illumination from shading) produces reasonable results on arbitrary grayscale images taken in the real world, and outperforms all previous grayscale “intrinsic image” - style algorithms on the MIT Intrinsic Images dataset.

[1]  D. Hilbert,et al.  Geometry and the Imagination , 1953 .

[2]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[3]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[4]  Berthold K. P. Horn,et al.  Determining lightness from an image , 1974, Comput. Graph. Image Process..

[5]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[6]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[7]  Alan L. Yuille,et al.  An Extremum Principle for Shape from Contour , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  J J Koenderink,et al.  What Does the Occluding Contour Tell Us about Solid Shape? , 1984, Perception.

[9]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[10]  Ramesh C. Jain,et al.  Segmentation through Variable-Order Surface Fitting , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Carlo H. Séquin,et al.  Functional optimization for fair surface design , 1992, SIGGRAPH.

[12]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[13]  J. Príncipe,et al.  Learning from examples with quadratic mutual information , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[14]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[15]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[16]  David Mumford,et al.  Statistics of range images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[18]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[19]  Zoubin Ghahramani,et al.  Optimization with EM and Expectation-Conjugate-Gradient , 2003, ICML.

[20]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[21]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[23]  A. Gilchrist.  Seeing in Black and White , 2006 .

[24]  David J. Kriegman,et al.  Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Cheng Lu,et al.  Entropy Minimization for Shadow Removal , 2009, International Journal of Computer Vision.

[27]  Edward H. Adelson,et al.  Ground truth dataset and baseline evaluations for intrinsic image algorithms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[28]  Jitendra Malik,et al.  High-frequency shape and albedo from shading using natural image statistics , 2011, CVPR 2011.

[29]  Xuelong Li,et al.  Intrinsic images using optimization , 2011, CVPR 2011.

[30]  Chuohao Yeo,et al.  Intrinsic images decomposition using a local and global sparse representation of reflectance , 2011, CVPR 2011.