Recovering intrinsic images from a single image

Interpreting real-world images requires the ability distinguish the different characteristics of the scene that lead to its final appearance. Two of the most important of these characteristics are the shading and reflectance of each point in the scene. We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, given the lighting direction, each image derivative is classified as being caused by shading or a change in the surface's reflectance. The classifiers gather local evidence about the surface's form and color, which is then propagated using the generalized belief propagation algorithm. The propagation step disambiguates areas of the image where the correct classification is not clear from local evidence. We use real-world images to demonstrate results and show how each component of the system affects the results.

[1]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[2]  A. Oppenheim,et al.  Nonlinear filtering of multiplied and convolved signals , 1968 .

[3]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[4]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[5]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[6]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[7]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Azriel Rosenfeld,et al.  Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System , 1985, Artif. Intell..

[9]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[10]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Mark S. Drew,et al.  Recovering Shading from Color Images , 1992, ECCV.

[12]  Edward H. Adelson,et al.  Recovering reflectance and illumination in a world of painted polyhedra , 1993, 1993 (4th) International Conference on Computer Vision.

[13]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[14]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[15]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[16]  Global Shape from Shading , 1995, Comput. Vis. Image Underst..

[17]  Paul A. Viola,et al.  Bayesian Model of Surface Perception , 1997, NIPS.

[18]  Ryszard Kozera.  An Overview of the shape-from-shading problem , 1998 .

[19]  Jitendra Malik,et al.  Recognizing surfaces using three-dimensional textons , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[22]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[23]  William T. Freeman,et al.  Learning Local Evidence for Shading and Reflectance , 2001, ICCV.

[24]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[25]  W. Freeman,et al.  Learning local evidence for shading and reflectance , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[27]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[28]  Yehezkel Yeshurun,et al.  A new perspective [on] shape-from-shading , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Olivier D. Faugeras,et al.  "Perspective shape from shading" and viscosity solutions , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[30]  J. M. Rubin,et al.  Color vision and image intensities: When are changes material? , 1982, Biological Cybernetics.

[31]  Ron Kimmel,et al.  Optimal Algorithm for Shape from Shading and Path Planning , 2001, Journal of Mathematical Imaging and Vision.

[32]  Katsushi Ikeuchi,et al.  Illumination normalization with time-dependent intrinsic images for video surveillance , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[34]  Alex Pentland.  Linear shape from shading , 2004, International Journal of Computer Vision.

[35]  Paul A. Viola,et al.  Boosting Image Retrieval , 2004, International Journal of Computer Vision.

[36]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[37]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.