Shape and Enlightenment: Reconstruction and Recognition under Variable Illumination

Due to illumination variability, the same object can appear dramatically different even when viewed in fixed pose, and this variability can confound recognition systems. This paper summarizes recent work on developing appearance-based methods for modeling the variability due to illumination in the images of objects. They differ from past appearance-based methods, however, in that a small set of training images is used to generate a representation — the illumination cone — which models the complete set of images of an object with Lambertian reflectance map under an arbitrary combination of point light sources at infinity. From a few images of an object in fixed pose but varying and unknown lighting, a surface and albedo map are reconstructed up to a family of affine (generalized bas-relief or GBR) deformations, and the cone representation is derived from this GBR surface. The methods have been tested within the domain of face recognition on two databases, one with 660 images of 10 faces in fixed pose but variable lighting, and one with 1350 images of 10 faces with variable pose and lighting; the results exceed those of popular existing methods.

[1]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[2]  S. Zucker,et al.  A ray-based computational model of light sources and illumination , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[3]  Friedrich Pfeiffer,et al.  A multistage-approach to the dynamics and control of elastic robots , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[4]  Daniel Snow,et al.  Shape and albedo from multiple images using integrability , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Steven Dubowsky,et al.  On the Dynamic Analysis and Behavior of Industrial Robotic Manipulators With Elastic Members , 1983 .

[6]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[7]  Gregory D. Hager,et al.  The confluence of vision and control, Block Island Workshop on Vision and Control, June 23-27, 1997, Block Island, Rhode Island, USA , 1998, Block Island Workshop on Vision and Control.

[8]  Tsuneo Yoshikawa,et al.  Modeling and Control of a Three Degree of Freedom Manipulator with Two Flexible Links , 1990, ISER.

[9]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[11]  David J. Kriegman,et al.  Illumination-based image synthesis: creating novel images of human faces under differing pose and lighting , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[12]  Atsushi Konno,et al.  Configuration-Dependent Controllability of Flexible Manipulators , 1993, ISER.

[13]  Masaru Uchiyama,et al.  Compensability of End-Effector Position Errors for Flexible Robot Manipulators , 1991, 1991 American Control Conference.

[14]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[15]  Kenichi Yoshimoto,et al.  Control of 3 dof spatial manipulator with a flexible link. , 1990 .

[16]  Atsushi Konno,et al.  Vibration suppression control of flexible robots using velocity inputs , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[17]  Masaru Uchiyama,et al.  Compensating Control of a Flexible Robot Arm , 1990, J. Robotics Mechatronics.

[18]  M. Uchiyama,et al.  Contribution to Inverse Kinematics of Flexible Robot Arms , 1994 .

[19]  Hiroshi Murase,et al.  Dimensionality of Illumination Manifolds in Appearance Matching , 1996, Object Representation in Computer Vision.

[20]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[21]  Hiroshi Murase,et al.  Subspace methods for robot vision , 1996, IEEE Trans. Robotics Autom..

[22]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  David J. Kriegman,et al.  Illumination cones for recognition under variable lighting: faces , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[25]  Russell A. Epstein,et al.  5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models , 1995, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[26]  Wayne J. Book,et al.  Feedback control of two beam, two joint systems with distributed flexibility , 1975 .

[27]  Atsushi Konno,et al.  Development of a flexible dual-arm manipulator testbed for space robotics , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[28]  Sabri Tosunoglu,et al.  Accessibility and Controllability of Flexible Robotic Manipulators , 1992 .

[29]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  David J. Kriegman,et al.  What shadows reveal about object structure. , 2001 .

[31]  Atsushi Konno,et al.  Vibration Suppressing Control of 3D Flexible Manipulators Using the Acceleration Command , 1994 .

[32]  L. Meirovitch.  Analytical Methods in Vibrations , 1967 .

[33]  R. H. Cannon,et al.  Initial Experiments on the End-Point Control of a Flexible One-Link Robot , 1984 .

[34]  Atsushi Konno,et al.  Vibration controllability of flexible manipulators , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.