Establishing Good Benchmarks and Baselines for Face Recognition

The Rowland Institute at Harvard, Harvard University, 100 Edwin Land Blvd.,Cambridge, MA 02142, USA,cox@rowland.harvard.eduAbstract. Progress in face recognition relies critically on the creationof test sets against which the performance of various approaches can beevaluated. A good set must capture the essential elements of what makesthe problem hard, while conforming to practical scale limitations. How-ever, these goals are often deceptively dicult to achieve. In the relatedarea of object recognition, Pinto et al. [2] demonstrated the potentialdangers of using a large, uncontrolled natural image set, showing thatan extremely rudimentary vision system (inspired by the early stages ofvisual processing in the brain) was able to perform on par with manystate-of-the-art vision systems on the popular Caltech101 object set [3].At the same time, this same rudimentary system was easily defeatedby an ostensibly \simpler" synthetic recognition test designed to betterspan the range of real world variation in object pose, position, scale, etc.These results suggested that image sets that look \natural" to human ob-servers may nonetheless fail to properly embody the problem of interest,and that care must be taken to establish baselines against which perfor-mance can be judged. Here, we repeat this approach for the \LabeledFaces in the Wild" (LFW) dataset [1], and for a collection of standardface recognition tests. The goal of the present work is not to competein the LFW challenge, per se, but to provide a baseline against whichthe performance of other systems can be judged. In particular, we foundthat our rudimentary \baseline" vision system was able to achieve  68%correct performance on the LFW challenge, substantially higher than apure \chance" baseline. We argue that this value might serve as a moreuseful baseline against which to evaluate absolute performance and arguethat the LFW set, while perhaps not perfect, represents an improvementover other standard face sets.