Quo vadis Face Recognition

Within the past decade, major advances have occurred in face recognition. With few exceptions, however, most research has been limited to training and testing on frontal views. Little is known about the extent to which face pose, illumination, expression, occlusion, and individual differences, such as those associated with gender, influence recognition accuracy. We systematically varied these factors to test the performance of two leading algorithms, one template based and the other feature based. Image data consisted of over 21000 images from 3 publicly available databases: CMU PIE, Cohn-Kanade, and AR databases. In general, both algorithms were robust to variation in illumination and expression. Recognition accuracy was highly sensitive to variation in pose. For frontal training images, performance was attenuated beginning at about 15 degrees. Beyond about 30 degrees, performance became unacceptable. For non-frontal training images, fall off was more severe. Small but consistent differences were found for individual differences in subjects. These findings suggest direction for future research, including design of experiments and data collection.