Likelihood Ratio as Weight of Forensic Evidence: A Closer Look.

The forensic science community has increasingly sought quantitative methods for conveying the weight of evidence. Experts from many forensic laboratories summarize their findings in terms of a likelihood ratio. Several proponents of this approach have argued that Bayesian reasoning proves it to be normative. We find this likelihood ratio paradigm to be unsupported by arguments of Bayesian decision theory, which applies only to personal decision making and not to the transfer of information from an expert to a separate decision maker. We further argue that decision theory does not exempt the presentation of a likelihood ratio from uncertainty characterization, which is required to assess the fitness for purpose of any transferred quantity. We propose the concept of a lattice of assumptions leading to an uncertainty pyramid as a framework for assessing the uncertainty in an evaluation of a likelihood ratio. We demonstrate the use of these concepts with illustrative examples regarding the refractive index of glass and automated comparison scores for fingerprints.

[1]  Didier Meuwly,et al.  Performance Study of a Score‐based Likelihood Ratio System for Forensic Fingermark Comparison , 2017, Journal of forensic sciences.

[2]  Suzanne O. Kaasa,et al.  Do Jurors Give Appropriate Weight to Forensic Identification Evidence? , 2013 .

[3]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[4]  Cedric Neumann,et al.  Quantifying the weight of evidence from a forensic fingerprint comparison: a new paradigm , 2012 .

[5]  B. Newell,et al.  On the interpretation of likelihood ratios in forensic science evidence: Presentation formats and the weak evidence effect. , 2014, Forensic science international.

[6]  Art B. Owen,et al.  Nonparametric Likelihood Confidence Bands for a Distribution Function , 1995 .

[7]  W. W. Peterson,et al.  The theory of signal detectability : Part I, the general theory : Part II, applications with Gaussian noise , 1953 .

[8]  Massimo Marinacci,et al.  Special Issue: Ninth International Symposium on Imprecise Probability: Theory and Applications (ISIPTA'15) , 2017, Int. J. Approx. Reason..

[9]  Robert C. Williamson,et al.  Probabilistic arithmetic. I. Numerical methods for calculating convolutions and dependency bounds , 1990, Int. J. Approx. Reason..

[10]  Scott Ferson,et al.  Constructing Probability Boxes and Dempster-Shafer Structures , 2003 .

[11]  L. J. Savage,et al.  Probability and the weighing of evidence , 1951 .

[12]  Jadhav Sachin Namdeo.  Generating, Classifying and Indexing Large Scale Fingerprints , 2012 .

[13]  Jeffrey N. Rouder,et al.  The philosophy of Bayes’ factors and the quantification of statistical evidence , 2016 .

[14]  A. P. Dawid.  Bayes's theorem and weighing evidence by juries , 2002 .

[15]  Ahmed H. Tewfik,et al.  Empirical Likelihood Ratio Test With Distribution Function Constraints , 2013, IEEE Transactions on Signal Processing.

[16]  Daniel Berleant,et al.  Arithmetic on Random Variables: Squeezing the Envelopes with New Joint Distribution Constraints , 2005, ISIPTA.

[17]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[18]  R. L. Winkler,et al.  Separating probability elicitation from utilities , 1988 .

[19]  Franco Taroni,et al.  Statistics and the Evaluation of Evidence for Forensic Scientists , 2004 .

[20]  I. Evett,et al.  The refractive index distribution of control glass samples examined by the Forensic Science Laboratories in the United Kingdom , 1984 .

[21]  F. O. Hoffman,et al.  Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. , 1994, Risk analysis : an official publication of the Society for Risk Analysis.

[22]  W. Thompson,et al.  Lay understanding of forensic statistics: Evaluation of random match probabilities, likelihood ratios, and verbal equivalents. , 2015, Law and human behavior.

[23]  James Michael Curran,et al.  Introduction to Data Analysis with R for Forensic Scientists , 2010 .

[24]  Daniel Berleant,et al.  Envelopes around cumulative distribution functions from interval parameters of standard continuous distributions , 2003, 22nd International Conference of the North American Fuzzy Information Processing Society, NAFIPS 2003.

[25]  David H. Kaye,et al.  Reference Guide on Statistics , 2011 .

[26]  A. Tversky,et al.  On the Reconciliation of Probability Assessments , 1979 .

[27]  J. A. Lambert,et al.  The interpretation of refractive index measurements VI: The computer program rung , 1985 .

[28]  C. Aitken,et al.  Liberties and constraints of the normative approach to evaluation and decision in forensic science: a discussion towards overcoming some common misconceptions , 2014 .

[29]  J M Curran,et al.  Spatial variation of refractive index in a pane of float glass. , 2003, Science & justice : journal of the Forensic Science Society.

[30]  H. Elffers,et al.  Understanding of forensic expert reports by judges, defense lawyers and forensic professionals , 2012 .

[31]  W. W. Peterson,et al.  The theory of signal detectability , 1954, Trans. IRE Prof. Group Inf. Theory.

[32]  E. S. Gillespie,et al.  The Evolving Role of Statistical Assessments as Evidence in the Courts. , 1990 .

[33]  Allan H. Seheult,et al.  On a problem in forensic science , 1978 .

[34]  Jesse Frey,et al.  Optimal distribution-free confidence bands for a distribution function , 2008 .

[35]  Colin Aitken,et al.  Dismissal of the illusion of uncertainty in the assessment of a likelihood ratio , 2016 .

[36]  Colin Aitken,et al.  Bayesian Networks and Probabilistic Inference in Forensic Science , 2006 .

[37]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[38]  R. Meester,et al.  Why the Effect of Prior Odds Should Accompany the Likelihood Ratio When Reporting DNA Evidence , 2004 .