Performance Generalization in Biometric Authentication Using Joint User-Specific and Sample Bootstraps

Biometric authentication performance is often depicted by a detection error trade-off (DET) curve. We show that this curve is dependent on the choice of samples available, the demographic composition and the number of users specific to a database. We propose a two-step bootstrap procedure to take into account the three mentioned sources of variability. This is an extension to the Bolle et al.'s bootstrap subset technique. Preliminary experiments on the NIST2005 and XM2VTS benchmark databases are encouraging, e.g., the average result across all 24 systems evaluated on NIST2005 indicates that one can predict, with more than 75 percent of DET coverage, an unseen DET curve with eight times more users. Furthermore, our finding suggests that with more data available, the confidence intervals become smaller and, hence, more useful

[1]  H. Hotelling Applications of the Theory of Error to the Interpretation of Trends , 1929 .

[2]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[3]  G. Campbell,et al.  Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. , 1994, Statistics in medicine.

[4]  Alvin F. Martin,et al.  The DET curve in assessment of detection task performance , 1997, EUROSPEECH.

[5]  Douglas A. Reynolds,et al.  SHEEP, GOATS, LAMBS and WOLVES A Statistical Analysis of Speaker Performance in the NIST 1998 Speaker Recognition Evaluation , 1998 .

[6]  James L. Wayman,et al.  Error rate equations for the general biometric system , 1999, IEEE Robotics Autom. Mag..

[7]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Sofus A. Macskassy,et al.  Confidence Bands for Roc Curves , 2003 .

[9]  Foster J. Provost,et al.  Confidence Bands for Roc Curves , 2004, ROCAI.

[10]  Samy Bengio,et al.  The expected performance curve: a new assessment measure for person authentication , 2004, Odyssey.

[11]  Sharath Pankanti,et al.  Error analysis of pattern recognition systems - the subsets bootstrap , 2004, Comput. Vis. Image Underst..

[12]  S. Buxbaum Sheep , 2004 .

[13]  M. Gamassi,et al.  Accuracy and performance of biometric systems , 2004, Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.04CH37510).

[14]  Samy Bengio,et al.  Compensating User-Specific Information with User-Independent Information in Biometric Authentication Tasks , 2005 .

[15]  Sarat C. Dass,et al.  Effects of user correlation on sample size requirements , 2005, SPIE Defense + Commercial Sensing.

[16]  Andy Adler,et al.  Calculation of a Composite DET Curve , 2005, AVBPA.

[17]  Foster J. Provost,et al.  ROC confidence bands: an empirical evaluation , 2005, ICML.

[18]  Samy Bengio,et al.  Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication , 2006, Pattern Recognit..

[19]  Samy Bengio,et al.  Estimating the Confidence Interval of Expected Performance Curve in Biometric Authentication using Joint Bootstrap , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[20]  L. Horváth,et al.  Confidence bands for ROC curves , 2008 .