Unconstrained face recognition: Establishing baseline human performance via crowdsourcing

Research focus in face recognition has shifted towards recognition of faces “in the wild” for both still images and videos which are captured in unconstrained imaging environments and without user cooperation. Due to confounding factors of pose, illumination, and expression, as well as occlusion and low resolution, current face recognition systems deployed in forensic and security applications operate in a semi-automatic manner; an operator typically reviews the top results from the face recognition system to manually determine the final match. For this reason, it is important to analyze the accuracies achieved by both the matching algorithms (machines) and humans on unconstrained face recognition tasks. In this paper, we report human accuracy on unconstrained faces in still images and videos via crowd-sourcing on Amazon Mechanical Turk. In particular, we report the first human performance on the YouTube Faces database and show that humans are superior to machines, especially when videos contain contextual cues in addition to the face image. We investigate the accuracy of humans from two different countries (United States and India) and find that humans from the United States are more accurate, possibly due to their familiarity with the faces of the public figures in the YouTube Faces database. A fusion of recognitions made by humans and a commercial-off-the-shelf face matcher improves performance over humans alone.

[1]  J. Brigham,et al.  Thirty years of investigating the own-race bias in memory for faces: A meta-analytic review , 2001 .

[2]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[3]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[5]  Anil K. Jain,et al.  A Case Study on Unconstrained Facial Recognition Using the Boston Marathon Bombings Suspects , 2013 .

[6]  V. Bruce,et al.  Matching identities of familiar and unfamiliar faces caught on CCTV images. , 2001, Journal of experimental psychology. Applied.

[7]  Neeraj Kumar,et al.  Describable Visual Attributes for Face Images , 2011 .

[8]  Rama Chellappa,et al.  Dictionary-Based Face Recognition from Video , 2012, ECCV.

[9]  Anil K. Jain,et al.  Face Matching and Retrieval in Forensics Applications , 2012, IEEE MultiMedia.

[10]  尚弘 島影.  National Institute of Standards and Technologyにおける超伝導研究及び生活 , 2001 .

[11]  Andy Adler,et al.  Comparing Human and Automatic Face Recognition Performance , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[13]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Alice J. O'Toole,et al.  Face Recognition Algorithms Surpass Humans Matching Faces Over Changes in Illumination , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Jean-Claude Martin,et al.  Voluntary facial activations regulate physiological arousal and subjective experiences during virtual social stimulation , 2012, TAP.

[16]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[17]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[18]  Anil K. Jain,et al.  Video-to-video face matching: Establishing a baseline for unconstrained face recognition , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[19]  Shree K. Nayar,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Describable Visual Attributes for Face Verification and Image Search , 2022 .

[20]  V. Bruce,et al.  Face Recognition in Poor-Quality Video: Evidence From Security Surveillance , 1999 .

[21]  Alice J. O'Toole,et al.  Comparing face recognition algorithms to humans on challenging tasks , 2012, TAP.

[22]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 Large-Scale Experimental Results , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.