Calculation of a Composite DET Curve

The verification performance of biometric systems is normally evaluated using the receiver operating characteristic (ROC) or detection error trade-off (DET) curve. We propose two new ideas for statistical evaluation of biometric systems based on these data. The first is a new way to normalize match score distributions. A normalized match score, $\hat{t}$, is calculated as a function of the angle from a representation of (FMR, FNMR) values in polar coordinates from some center. This has the advantage that it does not produce counterintuitive results for systems with unusual DET performance. Secondly, building on this normalization we develop a methodology to calculate an average DET curve. Each biometric system is represented in terms of $\hat{t}$ to allow genuine and impostor distributions to be combined, and an average DET is then calulated from these new distributions. We then show that this method is equivalent to direct averaging of DET data along each angle from the center. This procedure is then applied to data from a study of human matchers of facial images.

[1]  Matteo Golfarelli,et al.  On the Error-Reject Trade-Off in Biometric Verification Systems , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  J. Kardaun,et al.  Comparative Diagnostic Performance of Three Radiological Procedures for the Detection of Lumbar Disk Herniation , 1990, Methods of Information in Medicine.

[3]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[4]  Peter A. Flach,et al.  The 1st workshop on ROC analysis in artificial intelligence (ROCAI-2004) , 2004, SKDD.

[5]  V. Bruce,et al.  Human and automatic face recognition: a comparison across image formats , 2001, Vision Research.

[6]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Data Mining Researchers , 2003 .

[7]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[8]  Arun Ross,et al.  Score normalization in multimodal biometric systems , 2005, Pattern Recognit..

[9]  P. Jonathon Phillips,et al.  Dependence characteristics of face recognition algorithms , 2002, Object recognition supported by user interaction for service robots.

[10]  Robert C. Holte,et al.  What ROC Curves Can't Do (and Cost Curves Can) , 2004, ROCAI.

[11]  V. Bruce,et al.  A comparison of two computer-based face identification systems with human perceptions of faces , 1998, Vision Research.

[12]  Xiao-Hua Zhou,et al.  Statistical Methods in Diagnostic Medicine , 2002 .

[13]  Foster J. Provost,et al.  Confidence Bands for Roc Curves , 2004, ROCAI.

[14]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[15]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..