Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners

The interpretation of forensic fingerprint evidence relies on the expertise of latent print examiners. We tested latent print examiners on the extent to which they reached consistent decisions. This study assessed intra-examiner repeatability by retesting 72 examiners on comparisons of latent and exemplar fingerprints, after an interval of approximately seven months; each examiner was reassigned 25 image pairs for comparison, out of total pool of 744 image pairs. We compare these repeatability results with reproducibility (inter-examiner) results derived from our previous study. Examiners repeated 89.1% of their individualization decisions, and 90.1% of their exclusion decisions; most of the changed decisions resulted in inconclusive decisions. Repeatability of comparison decisions (individualization, exclusion, inconclusive) was 90.0% for mated pairs, and 85.9% for nonmated pairs. Repeatability and reproducibility were notably lower for comparisons assessed by the examiners as “difficult” than for “easy” or “moderate” comparisons, indicating that examiners' assessments of difficulty may be useful for quality assurance. No false positive errors were repeated (n = 4); 30% of false negative errors were repeated. One percent of latent value decisions were completely reversed (no value even for exclusion vs. of value for individualization). Most of the inter- and intra-examiner variability concerned whether the examiners considered the information available to be sufficient to reach a conclusion; this variability was concentrated on specific image pairs such that repeatability and reproducibility were very high on some comparisons and very low on others. Much of the variability appears to be due to making categorical decisions in borderline cases.

[1]  David Charlton,et al.  Why Experts Make Errors , 2006 .

[2]  R. A. Hicklin,et al.  Accuracy and reliability of forensic latent fingerprint decisions , 2011, Proceedings of the National Academy of Sciences.

[3]  Dale J. Prediger,et al.  Coefficient Kappa: Some Uses, Misuses, and Alternatives , 1981 .

[4]  R. Rust,et al.  Reliability Measures for Qualitative Data: Theory and Implications , 1994 .

[5]  J S Uebersax,et al.  Modeling approaches for the analysis of observer agreement. , 1992, Investigative radiology.

[6]  K. Gwet Kappa Statistic is not Satisfactory for Assessing the Extent of Agreement Between Raters , 2002 .

[7]  J. Bohan,et al.  Review of: Strengthening Forensic Science in the United States: A Path Forward , 2010 .

[8]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[9]  W. D. Perreault,et al.  Reliability of Nominal Data Based on Qualitative Judgments , 1989 .

[10]  Law. Policy Executive Summary of the National Academies of Science Reports, Strengthening Forensic Science in the United States: A Path Forward , 2009 .

[11]  W. D. Perreault,et al.  Reliability of Nominal Data Based on Qualitative Judgments , 1989 .

[12]  Pat A Wertheim,et al.  Testing for Potential Contextual Bias Effects During the Verification Stage of the ACE‐V Methodology when Conducting Fingerprint Comparisons * , 2009, Journal of forensic sciences.

[13]  C. Champod,et al.  The potential (negative) influence of observational biases at the analysis stage of fingermark individualisation. , 2007, Forensic science international.

[14]  Michael E. Schuckers Using the Beta-Binomial Distribution to Assess Performance of a Biometric Identification Device , 2003, Int. J. Image Graph..

[15]  William C. Thompson,et al.  The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion , 2002 .

[16]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[17]  REVISED DRAFT FOR COMMENT REVISED DRAFT FOR COMMENT STANDARDS FOR EXAMINING FRICTION RIDGE IMPRESSIONS AND RESULTING CONCLUSIONS , 2011 .

[18]  I. Dror,et al.  Contextual information renders experts vulnerable to making erroneous identifications. , 2006, Forensic science international.