What is the set of images of an object under all possible lighting conditions?

The appearance of a particular object depends on both the viewpoint from which it is observed and the light sources by which it is illuminated. If the appearance of two objects is never identical for any pose or lighting conditions, then-in theory - the objects can always be distinguished or recognized. The question arises: What is the set of images of an object under all lighting conditions and pose? In this paper, we consider only the set of images of an object under variable illumination (including multiple, extended light sources and attached shadows). We prove that the set of n-pixel images of a convex object with a Lambertian reflectance function, illuminated by an arbitrary number of point light sources at infinity, forms a convex polyhedral cone in IR/sup n/ and that the dimension of this illumination cone equals the number of distinct surface normals. Furthermore, we show that the cone for a particular object can be constructed from three properly chosen images. Finally, we prove that the set of n-pixel images of an object of any shape and with an arbitrary reflectance function, seen under all possible illumination conditions, still forms a convex cone in IR/sup n/. These results immediately suggest certain approaches to object recognition. Throughout this paper, we offer results demonstrating the empirical validity of the illumination cone representation.

[1]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[2]  E. Polak,et al.  Theory of optimal control and mathematical programming , 1969 .

[3]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[4]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[5]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[6]  Thomas O. Binford,et al.  Generic surface interpretation: observability model , 1988 .

[7]  Takeo Kanade,et al.  Image Segmentation And Reflection Analysis Through Color , 1988, Defense, Security, and Sensing.

[8]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[9]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[10]  David A. Forsyth,et al.  Reflections on Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  A. Shashua.  Geometry and Photometry in 3D Visual Recognition , 1992 .

[12]  H.D. Tagare,et al.  A framework for the construction of reflectance maps for machine vision , 1993 .

[13]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[16]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[17]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  S. Zucker,et al.  A ray-based computational model of light sources and illumination , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[19]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[20]  Hiroshi Murase,et al.  Dimensionality of illumination in appearance matching , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[21]  Hiroshi Murase,et al.  Dimensionality of Illumination Manifolds in Appearance Matching , 1996, Object Representation in Computer Vision.

[22]  Andrea J. van Doorn,et al.  Bidirectional Reflection Distribution Function Expressed in Terms of Surface Scattering Modes , 1996, ECCV.

[23]  David A. Forsyth,et al.  Body plans , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[26]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.