If the Shoe Fits They Might Acquit: The Value of Forensic Science Testimony

The probative value of forensic science evidence (such as a shoeprint) varies widely depending on how the evidence and hypothesis of interest is characterized. This paper uses a likelihood ratio (LR) approach to identify the probative value of forensic science evidence. It argues that the “evidence” component should be characterized as a “reported match,” and that the hypothesis component should be characterized as “the matching person or object is the source of the crime scene sample.” This characterization of the LR forces examiners to incorporate risks from sample mix-ups and examiner error into their match statistics. But will legal decision makers be sensitive to the extent to which examiners’ statistical testimony accounts for various potential sources of risk and error? A controlled experiment with 315 jury-eligible jurors who received a shoeprint match statistic in a hypothetical burglary case finds that, contrary to normative theory, people are more persuaded by statistical testimony that ignores various error risks than by testimony that is objectively stronger by virtue of taking those risks into account. The experiment also finds that jurors are relatively unresponsive to exposure of those risks by a defense attorney on cross-examination. These results support and extend previous research that finds many people are confused about how to evaluate the risk of error associated with expert forensic testimony.

[1]  Dale A. Nance,et al.  Juror Understanding of DNA Evidence: An Empirical Assessment of Presentation Formats for Trace Evidence with a Relatively Small Random‐Match Probability , 2005, The Journal of Legal Studies.

[2]  W. Black,et al.  Communicating the significance of radiologic test results: the likelihood ratio. , 1986, AJR. American journal of roentgenology.

[3]  D. Schum.  The Evidential Foundations of Probabilistic Reasoning , 1994 .

[4]  D. Schum,et al.  A Probabilistic Analysis of the Sacco and Vanzetti Evidence , 1996 .

[5]  D. Kahneman,et al.  Heuristics and Biases: The Psychology of Intuitive Judgment , 2002 .

[6]  Samuel Lindsey,et al.  The Random Match Probability (RMP) in DNA Evidence: Irrelevant and Prejudicial? , 1995 .

[7]  Statistics in the Jury Box: How Jurors Respond to Mitochondrial DNA Match Probabilities , 2007 .

[8]  Geoffrey Stewart Morrison,et al.  Forensic voice comparison and the paradigm shift. , 2009, Science & justice : journal of the Forensic Science Society.

[9]  G. Guyatt,et al.  Users ' Guides to the Medical Literature : III . How to Use an Article About a Diagnostic Test : A . Are the Results of the Study Valid ? , 2022 .

[10]  Bernard Robertson,et al.  Interpreting Evidence: Evaluating Forensic Science in the Courtroom , 1995 .

[11]  W. C. Farabee.  Indian Tribes of Eastern Peru , 2007 .

[12]  William C Thompson,et al.  Mock Jurors’ Reactions to Selective Presentation of Evidence from Multiple-Opportunity Searches , 2006, Law and human behavior.

[13]  R. A. Hicklin,et al.  Accuracy and reliability of forensic latent fingerprint decisions , 2011, Proceedings of the National Academy of Sciences.

[14]  William J. Bodziak,et al.  Footwear Impression Evidence: Detection, Recovery and Examination , 1999 .

[15]  J. Koehler.  On Conveying the Probative Value of DNA Evidence: Frequencies, Likelihood Ratios and Error Rates , 1996 .

[16]  Shari Seidman Diamond,et al.  Juror Reactions to DNA Evidence: Errors and Expectancies , 1999 .

[17]  Ian W. Evett,et al.  Case pre-assessment and review in a two-way transfer case , 1999 .

[18]  J. Koehler,et al.  The Psychology of Numbers in the Courtroom: How to Make DNA Match Statistics Seem Impressive or Insufficient , 2001 .

[19]  S. Cole,et al.  Psychological Aspects of Forensic Identification Evidence , 2007 .

[20]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[21]  Gordon H. Guyatt,et al.  Users' Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test A. Are the Results of the Study Valid? , 1994 .

[22]  Jie W Weiss,et al.  Bayesian Statistical Inference for Psychological Research , 2008 .

[23]  David H. Kaye,et al.  Can Jurors Understand Probabilistic Evidence , 1991 .

[24]  I. Evett,et al.  Evidence evaluation: a response to the court of appeal judgment in R v T. , 2011, Science & justice : journal of the Forensic Science Society.

[25]  Suzanne O. Kaasa,et al.  Statistical Inference and Forensic Evidence: Evaluating a Bullet Lead Match , 2007, Law and human behavior.

[26]  Matthieu Schmittbuhl,et al.  Probabilistic evaluation of handwriting evidence: likelihood ratio for authorship , 2008 .

[27]  W. Wagenaar.  A Bayesian Discussion of the Position of Expert Witnesses , 1988 .

[28]  Philipp Dahm,et al.  Evidence-Based Surgery Chirurgie factuelle Users ’ guides to the surgical literature : how to use an article about a diagnostic test , 2001 .

[29]  J. Koehler,et al.  The Misquantification of Probative Value , 2003, Law and human behavior.

[30]  I. Evett,et al.  A hierarchy of propositions: deciding which level to address in casework , 1998 .

[31]  J A Lambert,et al.  The impact of the principles of evidence interpretation on the structure and content of statements. , 2000, Science & justice : journal of the Forensic Science Society.

[32]  Michael J. Saks,et al.  Communicating opinion evidence in the forensic identification sciences: Accuracy and impact , 2008 .

[33]  William C. Thompson,et al.  Are Juries Competent to Evaluate Statistical Evidence , 1989 .

[34]  Laura Macchi,et al.  Thinking About Low-Probability Events , 2004, Psychological science.

[35]  William C. Thompson.  Subjective interpretation, laboratory error and the value of forensic DNA evidence: three case studies. , 1995 .

[36]  R. Zajonc,et al.  Affect, cognition, and awareness: affective priming with optimal and suboptimal stimulus exposures. , 1993, Journal of personality and social psychology.

[37]  F Taroni,et al.  A probabilistic approach to the joint evaluation of firearm evidence and gunshot residues. , 2006, Forensic science international.

[38]  Simon A. Cole,et al.  Forensics Without Uniqueness, Conclusions Without Individualization: The New Epistemology of Forensic Identification , 2009 .

[39]  Cedric Neumann,et al.  Quantitative assessment of evidential weight for a fingerprint comparison I. Generalisation to the comparison of a mark with set of ten prints from a suspect. , 2011, Forensic science international.

[40]  B. Fischhoff,et al.  Hypothesis Evaluation from a Bayesian Perspective. , 1983 .

[41]  Pierre Patenaude.  Modern scientific evidence , 2000 .

[42]  J. Koehler.  Error and Exaggeration in the Presentation of DNA Evidence at Trial , 1993 .

[43]  L. Ross,et al.  Human Inference: Strategies and Shortcomings of Social Judgment. , 1981 .

[44]  Didier Meuwly,et al.  Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Three Minutiæ , 2006, Journal of forensic sciences.

[45]  Jennifer L. Mnookin,et al.  The Need for a Research Culture in the Forensic Sciences , 2011 .

[46]  C Champod,et al.  Establishing the most appropriate databases for addressing source level propositions. , 2004, Science & justice : journal of the Forensic Science Society.

[47]  J. Koehler,et al.  When Are People Persuaded By DNA Match Statistics? , 2001, Law and human behavior.

[48]  Richard D. Friedman,et al.  Sometimes What Everybody Thinks They Know Is True , 2003, Law and human behavior.

[49]  C. Aitken,et al.  A Two‐Level Model for Evidence Evaluation , 2007, Journal of forensic sciences.

[50]  Philip Rose,et al.  An empirical estimate of the precision of likelihood ratios from a forensic-voice-comparison system. , 2011, Forensic science international.

[51]  Terance D. Miethe,et al.  Gold versus platinum: Do jurors recognize the superiority and limitations of DNA evidence compared to other types of forensic evidence? , 2008 .

[52]  Simon A. Cole,et al.  Grandfathering Evidence: Fingerprint Admissibility Rulings from Jennings to Llera Plaza and Back Again , 2004 .

[53]  F Taroni,et al.  Probabilistic reasoning in the law. Part 1: Assessment of probabilities and explanation of the value of DNA evidence. , 1998, Science & justice : journal of the Forensic Science Society.

[54]  R. Crepeau.  The Trial of O.J. Simpson , 1995 .

[55]  Ian W. Evett.  A Quantitative Theory for Interpreting Transfer Evidence in Criminal Cases , 1984 .

[56]  Franco Taroni,et al.  Statistics and the Evaluation of Evidence for Forensic Scientists , 2004 .

[57]  C. Aitken,et al.  Forensic science evidence in question , 2011 .

[58]  Franco Taroni,et al.  How the probability of a false positive affects the value of DNA evidence. , 2003, Journal of forensic sciences.

[59]  Michael S. Pardo,et al.  The Problematic Value of Mathematical Models of Evidence , 2007, The Journal of Legal Studies.

[60]  C. Aitken,et al.  Statistics and the Evaluation of Evidence for Forensic Scientists: Aitken/Statistics and the Evaluation of Evidence for Forensic Scientists , 2005 .

[61]  Michael J. Saks,et al.  The Testimony of Forensic Identification Science: What Expert Witnesses Say and What Factfinders Hear , 2009, Law and human behavior.

[62]  S. Varadhan,et al.  A probabilistic approach to , 1974 .

[63]  D. Balding.  Weight-of-Evidence for Forensic DNA Profiles , 2005 .

[64]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[65]  Dale A. Nance,et al.  An Empirical Assessment of Presentation Formats for Trace Evidence with a Relatively Large and Quantifiable Random Match Probability , 2002 .

[66]  Richard Hornby.  Beyond a Reasonable Doubt , 2005 .

[67]  Jonathan J. Koehler,et al.  Fingerprint Error Rates and Proficiency Tests: What They are and Why They Matter , 2008 .

[68]  G. Guyatt,et al.  Users' guides to the medical literature. , 1993, JAMA.

[69]  David H. Kaye.  Probability, Individualization, and Uniqueness in Forensic Science Evidence: Listening to the Academies , 2009 .

[70]  I. Evett,et al.  Interpreting DNA Evidence: Statistical Genetics for Forensic Scientists , 1998 .

[71]  Ian W. Evett,et al.  Interpreting DNA Evidence: A Review , 2003 .

[72]  I. Evett,et al.  More on the hierarchy of propositions: exploring the distinction between explanations and propositions. , 2000, Science & justice : journal of the Forensic Science Society.

[73]  Michael J. Saks,et al.  Human Information Processing and Adjudication: Trial by Heuristics , 1980 .

[74]  J A Lambert,et al.  A model for case assessment and interpretation. , 1998, Science & justice : journal of the Forensic Science Society.

[75]  Jonathan J. Koehler,et al.  The Individualization Fallacy in Forensic Science Evidence , 2008 .

[76]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[77]  Peter Neufeld,et al.  Invalid Forensic Science Testimony and Wrongful Convictions , 2009 .

[78]  Gerd Gigerenzer,et al.  Communicating statistical DNA evidence , 2003 .

[79]  Christophe Champod,et al.  Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Any Number of Minutiæ , 2007, Journal of forensic sciences.

[80]  Willem A. Wagenaar,et al.  The proper seat , 1988 .

[81]  J. Koehler,et al.  The Relevance Ratio: Evaluating the Probative Value of Expert Testimony in Child Sexual Abuse Cases , 1996 .

[82]  Jonathan J. Koehler,et al.  Individualization Claims in Forensic Science: Still Unwarranted , 2010 .

[83]  J. Frank Yates,et al.  Judgment and Decision Making , 1990 .

[84]  P. Meehl,et al.  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. , 1955, Psychological bulletin.

[85]  Jonathan J. Koehler,et al.  DNA Matches and Statistics: Important Questions, Surprising Answers , 1992 .