Face Recognition: The Problem of Compensating for Changes in Illumination Direction

A face recognition system must recognize a face from a novel image despite the variations between images of the same face. A common approach to overcoming image variations because of changes in the illumination conditions is to use image representations that are relatively insensitive to these variations. Examples of such representations are edge maps, image intensity derivatives, and images convolved with 2D Gabor-like filters. Here we present an empirical study that evaluates the sensitivity of these representations to changes in illumination, as well as viewpoint and facial expression. Our findings indicated that none of the representations considered is sufficient by itself to overcome image variations because of a change in the direction of illumination. Similar results were obtained for changes due to viewpoint and expression. Image representations that emphasized the horizontal features were found to be less sensitive to changes in the direction of illumination. However, systems based only on such representations failed to recognize up to 20 percent of the faces in our database. Humans performed considerably better under the same conditions. We discuss possible reasons for this superiority and alternative methods for overcoming illumination effects in recognition.

[1]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[2]  C Blakemore,et al.  On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images , 1969, The Journal of physiology.

[3]  P. O. Bishop,et al.  Spatial vision. , 1971, Annual review of psychology.

[4]  N. Graham,et al.  Detection of grating patterns containing two spatial frequencies: a comparison of single-channel and multiple-channels models. , 1971, Vision research.

[5]  B. Julesz,et al.  Spatial-frequency masking in vision: critical bands and spread of masking. , 1972, Journal of the Optical Society of America.

[6]  F. Werblin,et al.  Control of Retinal Sensitivity: I. Light and Dark Adaptation of Vertebrate Rods and Cones , 1974 .

[7]  F. Werblin.  Control of Retinal Sensitivity II. Lateral Interactions at the Outer Plexiform Layer , 1974 .

[8]  F. Werblin.  Control of Retinal Sensitivity II . Lateral Interactions at the Outer Plexiform Layer , 2022 .

[9]  Larry S. Davis,et al.  A survey of edge detection techniques , 1975 .

[10]  A. Baddeley,et al.  When face recognition fails. , 1977, Journal of experimental psychology. Human learning and memory.

[11]  H. Ellis,et al.  Face recognition accuracy as a function of mode of representation. , 1978 .

[12]  E. Warrington,et al.  Two Categorical Stages of Object Recognition , 1978, Perception.

[13]  J. Bergen,et al.  A four mechanism model for threshold spatial vision , 1979, Vision Research.

[14]  Takeo Kanade,et al.  Computer recognition of human faces , 1980 .

[15]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[16]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[17]  J. Shepherd.  Studies of cue saliency , 1981 .

[18]  D. Pollen,et al.  Phase relationships between adjacent simple cells in the visual cortex. , 1981, Science.

[19]  J. Robson,et al.  Discrimination at threshold: Labelled detectors in human vision , 1981, Vision Research.

[20]  V. Bruce.  Changing faces: visual and non-visual coding processes in face recognition. , 1982, British journal of psychology.

[21]  Daniel A. Pollen,et al.  Visual cortical neurons as localized spatial frequency filters , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[22]  G. Sandini,et al.  The Role of High Spatial Frequencies in Face Perception , 1983, Perception.

[23]  J. Daugman.  Spatial visual channels in the fourier plane , 1984, Vision Research.

[24]  Robert M. Haralick,et al.  Digital Step Edges from Zero Crossing of Second Directional Derivatives , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[26]  Tomaso A. Poggio,et al.  On Edge Detection , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  R. Haralick.  Digital Step Edges from Zero Crossing of Second Directional Derivatives , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  John Daugman.  Image Analysis And Compact Coding By Oriented 2D Gabor Primitives , 1987, Photonics West - Lasers and Applications in Science and Engineering.

[30]  Alice J. O'Toole,et al.  A physical system approach to recognition memory for spatially transformed faces , 1988, Neural Networks.

[31]  P.W.M. Tsang,et al.  A system for recognising human faces , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[32]  Venu Govindaraju,et al.  Locating human faces in newspaper photographs , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[34]  David W. Jacobs,et al.  Model group indexing for recognition , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Roberto Brunelli,et al.  HyperBF Networks for Real Object Recognition , 1991, IJCAI.

[36]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[37]  David W. Jacobs,et al.  Space and Time Bounds on Indexing 3D Models from 2D Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Yehezkel Yeshurun,et al.  Robust detection of facial features by generalized symmetry , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[40]  Edward M. Riseman,et al.  The non-existence of general-case view-invariants , 1992 .

[41]  Shimon Ullman,et al.  Limitations of Non Model-Based Recognition Schemes , 1992, ECCV.

[42]  B. S. Manjunath,et al.  A robust method for detecting image features with application to face recognition and motion correspondence , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[43]  Jun Shen,et al.  An optimal linear operator for step edge detection , 1992, CVGIP Graph. Model. Image Process..

[44]  David A. Forsyth,et al.  Extracting projective structure from single perspective views of 3D point sets , 1993, 1993 (4th) International Conference on Computer Vision.

[45]  Joachim M. Buhmann,et al.  A Silicon Retina for Face Recognition , 1993 .

[46]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[47]  T. Poggio,et al.  The importance of symmetry and virtual views in three-dimensional object recognition , 1994, Current Biology.

[48]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1995, Proceedings of IEEE International Conference on Computer Vision.

[49]  S. Ullman,et al.  Generalization to Novel Images in Upright and Inverted Faces , 1993, Perception.

[50]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.