Classification of Faces in Man and Machine

We attempt to shed light on the algorithms humans use to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. First, we apply principal component analysis (PCA) on the pixel information of the face stimuli. We then obtain a data set composed of these PCA eigenvectors combined with the subjects' gender estimates of the corresponding stimuli. Second, we model the gender classification process on this data set using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning: the support vector machine (SVM), the relevance vector machine, the prototype classifier, and the K-means classifier. The classification behavior of humans and machines is then analyzed in three steps. First, the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects' internal decision boundary by studying the training errors of the machines. Second, we study the correlations between the rank-order of the subjects' responses to each stimulusthe gender estimate with its reaction time and confidence ratingand the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects' gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans.Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based on stimuli close to the boundary between the classes.

[1]  Alice J. O'Toole,et al.  Face recognition algorithms as models of human face processing , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[2]  A. O'Toole,et al.  Prototype-referenced shape encoding revealed by high-level aftereffects , 2001, Nature Neuroscience.

[3]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[4]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[5]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[6]  Thomas J. Palmeri.  The time course of perceptual categorization , 2001, Similarity and Categorization.

[7]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[8]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[9]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[10]  R. Nosofsky.  Tests of an exemplar model for relating perceptual classification and recognition memory. , 1991, Journal of experimental psychology. Human perception and performance.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[13]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[14]  G. Cottrell,et al.  EMPATH: A Neural Network that Categorizes Facial Expressions , 2002, Journal of Cognitive Neuroscience.

[15]  Terrence J. Sejnowski,et al.  SEXNET: A Neural Network Identifies Sex From Human Faces , 1990, NIPS.

[16]  Thomas Vetter,et al.  Three-dimensional shape and two-dimensional surface reflectance contributions to face recognition: an application of three-dimensional morphing , 1999, Vision Research.

[17]  H. Abdi,et al.  Principal Component and Neural Network Analyses of Face Images: What Can Be Generalized in Gender Classification? , 1997, Journal of mathematical psychology.

[18]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[19]  T. Poggio,et al.  General conditions for predictivity in learning theory , 2004, Nature.

[20]  A. O'Toole,et al.  The perception of face gender: The role of stimulus structure in recognition and classification , 1998, Memory & cognition.

[21]  Wulfram Gerstner,et al.  Spiking Neuron Models: An Introduction , 2002 .

[22]  Shimon Edelman,et al.  Representation and recognition in vision , 1999 .

[23]  Felix A. Wichmann,et al.  Insights from Machine Learning Applied to Human Visual Classification , 2003, NIPS.

[24]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[25]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[26]  Andrew J. Calder,et al.  PII: S0042-6989(01)00002-5 , 2001 .

[27]  David Barber,et al.  Bayesian Classification With Gaussian Processes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Felix A. Wichmann,et al.  Gender Classification of Human Faces , 2002, Biologically Motivated Computer Vision.

[29]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[30]  Gustavo Deco,et al.  Computational neuroscience of vision , 2002 .

[31]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[32]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[33]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[34]  V. Bruce,et al.  A comparison of two computer-based face identification systems with human perceptions of faces , 1998, Vision Research.

[35]  Alice J. O'Toole,et al.  Face recognition algorithms and the other-race effect: computational mechanisms for a developmental contact hypothesis , 2002, Cogn. Sci..

[36]  Terrence J. Sejnowski,et al.  A Perceptron Reveals the Face of Sex , 1995, Neural Computation.

[37]  Shimon Edelman,et al.  Representation, similarity, and the chorus of prototypes , 1993, Minds and Machines.

[38]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[39]  H. Barlow.  Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[40]  F. Gregory Ashby,et al.  Categorization response time with multidimensional stimuli , 1994, Perception & psychophysics.

[41]  Peter Földiák,et al.  What is wrong with prototypes , 1998, Behavioral and Brain Sciences.

[42]  Stephen K. Reed,et al.  Pattern recognition and categorization , 1972 .

[43]  Garrison W. Cottrell,et al.  EMPATH: Face, Emotion, and Gender Recognition Using Holons , 1990, NIPS.

[44]  Thore Graepel,et al.  From Margin to Sparsity , 2000, NIPS.

[45]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[46]  Alexander J. Smola,et al.  Classification in a normalized feature space using support vector machines , 2003, IEEE Trans. Neural Networks.

[47]  Alice J. O'Toole,et al.  Low-dimensional representation of faces in higher dimensions of the face space , 1993 .

[48]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[49]  Seong-Whan Lee,et al.  Biologically Motivated Computer Vision , 2002, Lecture Notes in Computer Science.

[50]  T. Poggio,et al.  Neural mechanisms of object recognition , 2002, Current Opinion in Neurobiology.

[51]  Arnulf B. A. Graf.  Classification and feature extraction in man and machine , 2004 .

[52]  T. Wickens.  Elementary Signal Detection Theory , 2001 .

[53]  Bernhard Schölkopf,et al.  A tutorial on kernel methods for categorization , 2007, Journal of Mathematical Psychology.

[54]  Shawn W. Ell,et al.  The neurobiology of human category learning , 2001, Trends in Cognitive Sciences.

[55]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[56]  Bartlett W. Mel.  SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition , 1997, Neural Computation.

[57]  Lawrence Sirovich,et al.  On the Dimensionality of Face Space , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.