Estimating the Precision of the Likelihood-Ratio Output of a Forensic-Voice-Comparison System

The issues of validity and reliability are important in forensic science. Within the likelihood-ratio framework for the evaluation of forensic evidence, the log-likelihood-ratio cost (Cllr) has been applied as an appropriate metric for evaluating the accuracy of the output of a forensic-voice-comparison system, but there has been little research on developing a quantitative metric of precision. The present paper describes two procedures for estimating the precision of the output of a forensic-comparison system, a non-parametric estimate and a parametric estimate of its 95% credible interval. The procedures are applied to estimate the precision of a basic automatic forensic-voice-comparison system presented with different amounts of questioned-speaker data. The importance of considering precision is discussed.

[1]  Sadaoki Furui,et al.  Speaker-independent isolated word recognition using dynamic features of speech spectrum , 1986, IEEE Trans. Acoust. Speech Signal Process..

[2]  Didier Meuwly,et al.  The inference of identity in forensic speaker recognition , 2000, Speech Commun..

[3]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[4]  Franco Taroni,et al.  Statistics and the Evaluation of Evidence for Forensic Scientists , 2004 .

[5]  Alvin F. Martin,et al.  The NIST speaker recognition evaluation program , 2005 .

[6]  K. Koch.  Introduction to Bayesian Statistics , 2007 .

[7]  Standards for the formulation of evaluative forensic science expert opinion. , 2009, Science & justice : journal of the Forensic Science Society.

[8]  J. Koehler,et al.  The Coming Paradigm Shift in Forensic Identification Science , 2005, Science.

[9]  Cedric Charles Gilson,et al.  The Admissibility of Expert Evidence in Criminal Proceedings in England and Wales , 2010 .

[10]  Geoffrey Stewart Morrison,et al.  Forensic voice comparison and the paradigm shift. , 2009, Science & justice : journal of the Forensic Science Society.

[11]  John Buckleton.  A Framework for Interpreting Evidence , 2004 .

[12]  J M Curran,et al.  Assessing uncertainty in DNA evidence caused by sampling effects. , 2002, Science & justice : journal of the Forensic Science Society.

[13]  Geoffrey Stewart Morrison,et al.  A comparison of procedures for the calculation of forensic likelihood ratios from acoustic-phonetic data: Multivariate kernel density (MVKD) versus Gaussian mixture model-universal background model (GMM-UBM) , 2011, Speech Commun..

[14]  Philip Rose,et al.  An empirical estimate of the precision of likelihood ratios from a forensic-voice-comparison system. , 2011, Forensic science international.

[15]  Claude Roux,et al.  Statistics and the Evaluation of Evidence for Forensic Scientists, by Colin G. G. Aitken and Franco Taroni 2nd edition. John Wiley and Sons, 2004. , 2006 .

[16]  Javier Ortega-Garcia,et al.  Robust estimation, interpretation and assessment of likelihood ratios in forensic speaker recognition , 2006, Comput. Speech Lang..

[17]  Doroteo Torre Toledano,et al.  Emulating DNA: Rigorous Quantification of Evidential Weight in Transparent and Testable Forensic Speaker Recognition , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  Sridha Sridharan,et al.  Feature warping for robust speaker verification , 2001, Odyssey.

[19]  David A. van Leeuwen,et al.  An Introduction to Application-Independent Evaluation of Speaker Recognition Systems , 2007, Speaker Classification.

[20]  Phil Rose,et al.  A response to the UK position statement on forensic speaker comparison , 2009 .

[21]  Julie Zhou.  Robust Estimationに , 2009 .

[22]  Des Spence.  Interpreting the evidence , 2002, BMJ : British Medical Journal.

[23]  B. D. Gaudette.  The use of statistics in forensic science , 1987 .

[24]  I. W. Evett,et al.  Towards a uniform framework for reporting opinions in forensic science casework , 1998 .

[25]  Niko Brümmer,et al.  Application-independent evaluation of speaker detection , 2006, Comput. Speech Lang..

[26]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[27]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[28]  S. Sinha.  Introduction to Bayesian Statistics (2nd ed.) , 2008 .

[29]  Madhuri S. Mulekar,et al.  Weight-of Evidence for Forensic DNA Profiles , 2008, Technometrics.

[30]  Stationery Office.  The admissibility of expert evidence in criminal proceedings in England and Wales : a new approach to the determination of evidentiary reliability : a consultation paper , 2009 .

[31]  J. Curran.  An introduction to Bayesian credible intervals for sampling error in DNA profiles , 2005 .

[32]  C. Barden,et al.  Proficiency Testing Trends Following the 2009 National Academy of Sciences Report, “Strengthening Forensic Science in the United States: A Path Forward” , 2016 .

[33]  I. Evett,et al.  A hierarchy of propositions: deciding which level to address in casework , 1998 .

[34]  G. Morrison.  Likelihood-ratio forensic voice comparison using parametric representations of the formant trajectories of diphthongs. , 2009, The Journal of the Acoustical Society of America.

[35]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[36]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[37]  Hans C. van Houwelingen,et al.  The Elements of Statistical Learning, Data Mining, Inference, and Prediction. Trevor Hastie, Robert Tibshirani and Jerome Friedman, Springer, New York, 2001. No. of pages: xvi+533. ISBN 0‐387‐95284‐5 , 2004 .