Physiological and behavioral lip biometrics: A comprehensive study of their discriminative power

Compared with other traditional biometric features such as face, fingerprint, or handwriting, lip biometric features contain both physiological and behavioral information. Physiologically, different people have different lips. On the other hand, people can usually be differentiated by their talking style. Current research on lip biometrics generally does not distinguish between the two kinds of information during feature extraction and classification and the interesting question of whether the physiological or the behavioral lip features are more discriminative has not been comprehensively studied. In this paper, different physiological and behavioral lip features are studied with respect to their discriminative power in speaker identification and verification. Our experimental results have shown that both the static lip texture feature and the dynamic shape deformation feature can achieve high identification accuracy (above 90%) and low verification error rate (below 5%). In addition, the lip rotation and centroid deformations, which are related to the speaker's talking mannerism, are found to be useful for speaker identification and verification. In contrast to previous studies, our results show that behavioral lip features are more discriminative in speaker identification and verification compared to physiological features.

[1]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[2]  A. Murat Tekalp,et al.  Joint audio-video processing for biometric speaker identification , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[3]  Michal Choras,et al.  The lip as a biometric , 2010, Pattern Analysis and Applications.

[4]  John S. D. Mason,et al.  The role of dynamics in visual speech biometrics , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Timothy F. Cootes,et al.  Extraction of Visual Features for Lipreading , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Alan Wee-Chung Liew,et al.  A new optimization procedure for extracting the point-based lip contour using active shape model , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[7]  Alan Wee-Chung Liew,et al.  ICA-Based Lip Feature Representation for Speaker Authentication , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.

[8]  Juergen Luettin,et al.  Speaker identification by lipreading , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[9]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[10]  Xin Li,et al.  Speaker identification using speech and lip features , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[11]  D. Reynolds,et al.  Authentication gets personal with biometrics , 2004, IEEE Signal Processing Magazine.

[12]  Juergen Luettin,et al.  Learning to recognise talking faces , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[13]  Alan Wee-Chung Liew,et al.  Robust lip region segmentation for lip images with complex background , 2007, Pattern Recognit..

[14]  X. Zhang,et al.  Automatic speechreading with application to speaker verification , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[15]  A. Murat Tekalp,et al.  Discriminative Analysis of Lip Motion Features for Speaker Identification and Speech-Reading , 2006, IEEE Transactions on Image Processing.

[16]  Sridha Sridharan,et al.  The use of temporal speech and lip information for multi-modal speaker identification via multi-stream HMMs , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[17]  Yochai Konig,et al.  "Eigenlips" for robust speech recognition , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  S. Sridharan,et al.  A syntactic approach to automatic lip feature extraction for speaker identification , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[19]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[20]  Shu Hung Leung,et al.  Lip features selection with application to person authentication , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.