Deriving intrinsic images from image sequences

Intrinsic images are a useful midlevel description of scenes proposed by H.G. Barrow and J.M. Tenenbaum (1978). An image is de-composed into two images: a reflectance image and an illumination image. Finding such a decomposition remains a difficult problem in computer vision. We focus on a slightly, easier problem: given a sequence of T images where the reflectance is constant and the illumination changes, can we recover T illumination images and a single reflectance image? We show that this problem is still imposed and suggest approaching it as a maximum-likelihood estimation problem. Following recent work on the statistics of natural images, we use a prior that assumes that illumination images will give rise to sparse filter outputs. We show that this leads to a simple, novel algorithm for recovering reflectance images. We illustrate the algorithm's performance on real and synthetic image sequences.

[1]  Journal of the Optical Society of America , 1950, Nature.

[2]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[3]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[4]  Mark S. Drew,et al.  Recovering Shading from Color Images , 1992, ECCV.

[5]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[6]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[7]  Eero P. Simoncelli.  Statistical models for images: compression, restoration and synthesis , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[8]  Paul A. Viola,et al.  Bayesian Model of Surface Perception , 1997, NIPS.

[9]  Harry Shum,et al.  Construction and refinement of panoramic mosaics with global and local alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  H Farid,et al.  Separating reflections from images by use of independent component analysis. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[11]  David J. Kriegman,et al.  From few to many: generative models for recognition under variable pose and illumination , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[12]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  Richard Szeliski,et al.  Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001 .