What Shadows Reveal about Object Structure

In a scene observed from a fixed viewpoint, the set of shadow boundaries in an image changes as a point light source (nearby or at infinity) assumes different locations. We show that for any finite set of point light sources illuminating an object viewed under either orthographic or perspective projection, there is an equivalence class of object shapes having the same set of shadows. Members of this equivalence class differ by a four-parameter family of projective transformations, and the shadows of a transformed object are identical when the same transformation is applied to the light source locations. Under orthographic projection, this family is the generalized bas-relief (GBR) transformation, and we show that the GBR transformation is the only family of transformations of an object's shape for which the complete set of imaged shadows is identical. Finally, we show that given multiple images under differing and unknown light source directions, it is possible to reconstruct both an object's surface and the light source locations up to this family of transformations from the shadows alone.

[1]  H. Piaggio.  Differential Geometry of Curves and Surfaces , 1952, Nature.

[2]  J. Davenport.  Editor , 1960 .

[3]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[4]  David L. Waltz,et al.  Understanding Line drawings of Scenes with Shadows , 1975 .

[5]  Manfredo P. do Carmo,et al.  Differential geometry of curves and surfaces , 1976 .

[6]  Andrew P. Within.  Intensity-based edge classification , 1982, AAAI 1982.

[7]  Andrew P. Witkin,et al.  Intensity-Based Edge Classification , 1982, AAAI.

[8]  Takeo Kanade,et al.  Using shadows in finding surface orientations , 1983, Comput. Vis. Graph. Image Process..

[9]  Ramakant Nevatia,et al.  Detection of Buildings in Aerial Images Using Shape and Shadows , 1983, IJCAI.

[10]  B. F. Cook.  The Elgin Marbles , 1984 .

[11]  Murray H. Loew,et al.  The Entry-Exit Method of Shadow Boundary Segmentation , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[13]  R. Bruce Irvin,et al.  Methods for exploiting the relationship between buildings and their shadows in aerial imagery , 1989, IEEE Trans. Syst. Man Cybern..

[14]  Michael Hatzitheodorou,et al.  The derivation of 3-D surface shape from shadows , 1989 .

[15]  E. Olszewski,et al.  Leonardo on Painting. , 1990 .

[16]  M. Kemp.  The science of art : Optical themes in western art from Brunelleschi to Seurat , 1991 .

[17]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[18]  Martin Kemp,et al.  The Science of Art: Optical Themes in Western Art from Brunelleschi to Seurat by Martin Kemp (review) , 1991, Technology and Culture.

[19]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  A. Shashua.  Geometry and Photometry in 3D Visual Recognition , 1992 .

[21]  Claire-Lise Mottaz Jiang,et al.  Shadow segmentation and classification in a constrained environment , 1994 .

[22]  F. Cheng,et al.  Delimiting the building heights in a city from the shadow in a panchromatic spot-image , 1995 .

[23]  O. Faugeras.  Strati cation of D vision projective a ne and metric representations , 1995 .

[24]  Emanuele Trucco,et al.  Geometric Invariance in Computer Vision , 1995 .

[25]  O. Faugeras.  Stratification of three-dimensional vision: projective, affine, and metric representations , 1995 .

[26]  Steven W. Zucker,et al.  Shadows and shading flow fields , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Jan J. Koenderink,et al.  Affine structure and photometry , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Lawrence B. Wolff,et al.  Surface Curvature and Shape Reconstruction from Unknown Multiple Illumination and Integrability , 1997, Comput. Vis. Image Underst..

[30]  Steven W. Zucker,et al.  What is a light source? , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Ian D. Reid,et al.  Duality, Rigidity and Planar Parallax , 1998, ECCV.

[32]  Gregory Dudek,et al.  On 3-D surface reconstruction using shape from shadows , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).