Fusion of Talking Face Biometric Modalities for Personal Identity Verification

We describe a personal identity verification system based on lip dynamics biometric. The lip shape is represented in terms of a B-spline model, tracked over time. The coordinates of the 11 control points of the B-spline model are used as features for each frame. An utterance consisting of N frames produces a sequence of 22 dimensional feature vectors that is matched to the template using dynamic time warping. The verification error rate achieved by the systems on the XM2VTS database is about 14%. By fusing the system with face and voice biometrics the error rate is reduced to a fraction of one percent

[1]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Juergen Luettin,et al.  Visual Speech and Speaker Recognition , 1997 .

[3]  Jiri Matas,et al.  Fast face localisation and verification , 1999, Image Vis. Comput..

[4]  Josef Kittler,et al.  Feature selection for a DTW-based speaker verification system , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[5]  Juergen Luettin,et al.  Acoustic-labial speaker verification , 1997, Pattern Recognit. Lett..

[6]  James L. Wayman,et al.  Error rate equations for the general biometric system , 1999, IEEE Robotics Autom. Mag..

[7]  Jiri Matas,et al.  XM2VTSDB: The Extended M2VTS Database , 1999 .

[8]  Luc Vandendorpe,et al.  The M2VTS Multimodal Face Database (Release 1.00) , 1997, AVBPA.

[9]  Jiri Matas,et al.  Effective Implementation of Linear Discriminant Analysis for Face Recognition and Verification , 1999, CAIP.

[10]  Jiri Matas,et al.  Audio-visual person verification , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[11]  Anastasios Tefas,et al.  Multi modal verification for teleservices and security applications (M2VTS) , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[12]  Roland Auckenthaler,et al.  Lip signatures for automatic person recognition , 1999, 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No.99TH8451).

[13]  Jiri Matas,et al.  Lip-shape Dependent Face Verification , 1997, AVBPA.