An evaluation of multimodal 2D+3D face biometrics

We report on the largest experimental study to date in multimodal 2D+3D face recognition, involving 198 persons in the gallery and either 198 or 670 time-lapse probe images. PCA-based methods are used separately for each modality and match scores in the separate face spaces are combined for multimodal recognition. Major conclusions are: 1) 2D and 3D have similar recognition performance when considered individually, 2) combining 2D and 3D results using a simple weighting scheme outperforms either 2D or 3D alone, 3) combining results from two or more 2D images using a similar weighting scheme also outperforms a single 2D image, and 4) combined 2D+3D outperforms the multi-image 2D result. This is the first (so far, only) work to present such an experimental control to substantiate multimodal performance improvement.

[1]  Patrick J. Flynn,et al.  Using multiple gallery and probe images per person to improve performance of face recognition , 2003 .

[2]  Michael G. Strintzis,et al.  Integration of 2D and 3D images for enhanced face authentication , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[3]  Marc Acheroy,et al.  Face verification from 3D and grey level clues , 2001, Pattern Recognit. Lett..

[4]  Xin Chen,et al.  Multi-biometrics using facial appearance, shape and temperature , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[5]  Bruce A. Draper,et al.  Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures , 2003 .

[6]  Chin-Seng Chua,et al.  Facial feature detection and face recognition from 2D and 3D images , 2002, Pattern Recognit. Lett..

[7]  Patrick J. Flynn,et al.  Face Recognition Using 2D and 3D Facial Data , 2003 .

[8]  Patrick J. Flynn,et al.  A survey of approaches to three-dimensional face recognition , 2004, ICPR 2004.

[9]  Patrick J. Flynn,et al.  A Survey Of 3D and Multi-Modal 3D+2D Face Recognition , 2004 .

[10]  Shihong Lao,et al.  3D template matching for pose invariant face recognition using 3D facial model built with isoluminance line based stereo vision , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[11]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Bruce A. Draper,et al.  A Statistical Assessment of Subject Factors in the PCA Recognition of Human Faces , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[13]  Samy Bengio,et al.  A multi-sample multi-source model for biometric authentication , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.