What Is the Set of Images of an Object Under All Possible Illumination Conditions?

The appearance of an object depends on both the viewpoint from which it is observed and the light sources by which it is illuminated. If the appearance of two objects is never identical for any pose or lighting conditions, then–in theory–the objects can always be distinguished or recognized. The question arises: What is the set of images of an object under all lighting conditions and pose? In this paper, we consider only the set of images of an object under variable illumination, including multiple, extended light sources and shadows. We prove that the set of n-pixel images of a convex object with a Lambertian reflectance function, illuminated by an arbitrary number of point light sources at infinity, forms a convex polyhedral cone in IRn and that the dimension of this illumination cone equals the number of distinct surface normals. Furthermore, the illumination cone can be constructed from as few as three images. In addition, the set of n-pixel images of an object of any shape and with a more general reflectance function, seen under all possible illumination conditions, still forms a convex cone in IRn. Extensions of these results to color images are presented. These results immediately suggest certain approaches to object recognition. Throughout, we present results demonstrating the illumination cone representation.

[1]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[2]  E. Polak,et al.  Theory of optimal control and mathematical programming , 1969 .

[3]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[4]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[5]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[6]  Thomas O. Binford,et al.  Generic surface interpretation: observability model , 1988 .

[7]  Takeo Kanade,et al.  Image Segmentation And Reflection Analysis Through Color , 1988, Defense, Security, and Sensing.

[8]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[9]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[10]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  David A. Forsyth,et al.  Reflections on Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  A. Shashua.  Geometry and Photometry in 3D Visual Recognition , 1992 .

[13]  S. Ullman,et al.  Geometry and photometry in three-dimensional visual recognition , 1993 .

[14]  H.D. Tagare,et al.  A framework for the construction of reflectance maps for machine vision , 1993 .

[15]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[17]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  S. Zucker,et al.  A ray-based computational model of light sources and illumination , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[20]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[21]  Hiroshi Murase,et al.  Dimensionality of illumination in appearance matching , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[22]  Andrea J. van Doorn,et al.  Bidirectional Reflection Distribution Function Expressed in Terms of Surface Scattering Modes , 1996, ECCV.

[23]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  David A. Forsyth,et al.  Body plans , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[27]  David W. Jacobs,et al.  Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Thomas de Quincey.  [C] , 2000, The Works of Thomas De Quincey, Vol. 1: Writings, 1799–1820.

[30]  Sang Wook Lee,et al.  Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation , 1996, International Journal of Computer Vision.

[31]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[32]  Takeo Kanade,et al.  Shape from interreflections , 2004, International Journal of Computer Vision.

[33]  Shree K. Nayar,et al.  Generalization of the Lambertian model and implications for machine vision , 1995, International Journal of Computer Vision.

[34]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.