Color Constancy, Intrinsic Images, and Shape Estimation

We present SIRFS (shape, illumination, and reflectance from shading), the first unified model for recovering shape, chromatic illumination, and reflectance from a single image. Our model is an extension of our previous work [1], which addressed the achromatic version of this problem. Dealing with color requires a modified problem formulation, novel priors on reflectance and illumination, and a new optimization scheme for dealing with the resulting inference problem. Our approach outperforms all previously published algorithms for intrinsic image decomposition and shape-from-shading on the MIT intrinsic images dataset [1, 2] and on our own "naturally" illuminated version of that dataset.

[1]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[2]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[3]  Chuohao Yeo,et al.  Intrinsic images decomposition using a local and global sparse representation of reflectance , 2011, CVPR 2011.

[4]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[5]  Huang Yumin,et al.  A PHYSICAL APPROACH TO COLOR IMAGE UNDERSTANDING , 1991 .

[6]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Xuelong Li,et al.  Intrinsic images using optimization , 2011, CVPR 2011.

[8]  Katja Doerschner,et al.  Surface color perception in three-dimensional scenes , 2006, Visual Neuroscience.

[9]  David J. Kriegman,et al.  Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[11]  David A. Forsyth,et al.  A novel algorithm for color constancy , 1990, International Journal of Computer Vision.

[12]  Berthold K. P. Horn,et al.  Determining lightness from an image , 1974, Comput. Graph. Image Process..

[13]  Cheng Lu,et al.  Entropy Minimization for Shadow Removal , 2009, International Journal of Computer Vision.

[14]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[16]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Jitendra Malik,et al.  Shape, albedo, and illumination from a single image of an unknown object , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jitendra Malik,et al.  High-frequency shape and albedo from shading using natural image statistics , 2011, CVPR 2011.

[19]  Demetri Terzopoulos,et al.  Image Analysis Using Multigrid Relaxation Methods , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  L. Maloney,et al.  Color constancy: a method for recovering surface spectral reflectance , 1987 .

[21]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  A. Gilchrist.  Seeing in Black and White , 2006 .

[23]  Allen R. Hanson,et al.  Computer Vision Systems , 1978 .

[24]  M. Werman,et al.  Color lines: image specific color representation , 2004, CVPR 2004.

[25]  Jiawen Chen,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, SIGGRAPH 2007.

[26]  Edward H. Adelson,et al.  Ground truth dataset and baseline evaluations for intrinsic image algorithms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.

[28]  J. Príncipe,et al.  Learning from examples with quadratic mutual information , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[29]  Peter V. Gehler,et al.  Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance , 2011, NIPS.