Validating a Biometric Authentication System: Sample Size Requirements

Authentication systems based on biometric features (e.g., fingerprint impressions, iris scans, human face images, etc.) are increasingly gaining widespread use and popularity. Often, vendors and owners of these commercial biometric systems claim impressive performance that is estimated based on some proprietary data. In such situations, there is a need to independently validate the claimed performance levels. System performance is typically evaluated by collecting biometric templates from n different subjects, and for convenience, acquiring multiple instances of the biometric for each of the n subjects. Very little work has been done in 1) constructing confidence regions based on the ROC curve for validating the claimed performance levels and 2) determining the required number of biometric samples needed to establish confidence regions of prespecified width for the ROC curve. To simplify the analysis that addresses these two problems, several previous studies have assumed that multiple acquisitions of the biometric entity are statistically independent. This assumption is too restrictive and is generally not valid. We have developed a validation technique based on multivariate copula models for correlated biometric acquisitions. Based on the same model, we also determine the minimum number of samples required to achieve confidence bands of desired width for the ROC curve. We illustrate the estimation of the confidence bands as well as the required number of biometric samples using a fingerprint matching system that is applied on samples collected from a small population

[1]  E. Luciano,et al.  Copula methods in finance , 2004 .

[2]  James L. Wayman,et al.  Technical Testing and Evaluation of Biometric Identification Devices , 1996 .

[3]  J. L. Wayman,et al.  Best practices in testing and reporting performance of biometric devices. , 2002 .

[4]  W. J. Langford.  Statistical Methods , 1959, Nature.

[5]  Anil K. Jain,et al.  On-line fingerprint verification , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[6]  Michael E. Schuckers.  Using the Beta-Binomial Distribution to Assess Performance of a Biometric Identification Device , 2003, Int. J. Image Graph..

[7]  N. L. Johnson,et al.  Linear Statistical Inference and Its Applications , 1966 .

[8]  Sharath Pankanti,et al.  Guide to Biometrics , 2003, Springer Professional Computing.

[9]  T. Caliński,et al.  Linear Statistical Inference , 1985 .

[10]  Bill Ravens,et al.  An Introduction to Copulas , 2000, Technometrics.

[11]  Sharath Pankanti,et al.  Error analysis of pattern recognition systems - the subsets bootstrap , 2004, Comput. Vis. Image Underst..

[12]  William J. Wilson,et al.  Multivariate Statistical Methods , 2005, Technometrics.

[13]  Calyampudi Radhakrishna Rao,et al.  Linear Statistical Inference and its Applications , 1967 .

[14]  Bruce A. Draper,et al.  A nonparametric statistical comparison of principal component and linear discriminant subspaces for face recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  Sharath Pankanti,et al.  Evaluation techniques for biometrics-based authentication systems (FRR) , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[16]  E. Luciano,et al.  Copula Methods in Finance: Cherubini/Copula , 2004 .

[17]  Charles E. Heckler,et al.  Applied Multivariate Statistical Analysis , 2005, Technometrics.

[18]  Terrance E. Boult,et al.  Efficient evaluation of classification and recognition systems , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.