Another look at the (im-)precision of individual risk estimates made using actuarial risk assessment instruments.

We investigated the precision of individual risk estimates made using actuarial risk assessment instruments (ARAIs) by discussing some major conceptual issues and then illustrating them by analyzing new data. We used a standard multivariate statistical procedure, logistic regression, to create a new ARAI based on data from a follow-up study of 90 adult male sex offenders. We indexed predictive precision at the group level using confidence intervals for group mean probability estimates, and at the individual level using prediction intervals for individual probability estimates. Consistent with past research, ARAI scores were moderately and significantly predictive of failure in the aggregate, but group probability estimates had substantial margins of error and individual probability estimates had very large margins of error. We conclude that, without major advances in our understanding of the causes of violence, ARAIs cannot be used to estimate the specific probability or absolute likelihood of future violence with any reasonable degree of precision or certainty. The implications for conducting violence risk assessments in forensic mental health are discussed.

[1]  D G Altman,et al.  What do we mean by validating a prognostic model? , 2000, Statistics in medicine.

[2]  S. Hart,et al.  Precision of actuarial risk assessment instruments: evaluating the 'margins of error' of group v. individual predictions of violence. , 2007, The British journal of psychiatry. Supplement.

[3]  E. B. Wilson.  Probable Inference, the Law of Succession, and Statistical Inference , 1927 .

[4]  David Thornton,et al.  Predicting Recidivism Amongst Sexual Offenders: A Multi-site Study of Static-2002 , 2010, Law and human behavior.

[5]  P. Meehl,et al.  Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. , 1996 .

[6]  Catherine A. Cormier,et al.  Violent offenders: Appraising and managing risk, 2nd ed. , 2006 .

[7]  Jon R. Sorensen,et al.  An Actuarial Model for Assessment of Prison Violence Risk Among Maximum Security Inmates , 2005, Assessment.

[8]  Leam A. Craig,et al.  Best practice in conducting actuarial risk assessments with adult sexual offenders , 2009 .

[9]  D. Doren.  Evaluating Sex Offenders , 2002 .

[10]  A. Agresti,et al.  Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions , 1998 .

[11]  Forensic Use of Actuarial Risk Assessment with Sex Offenders: Accuracy, Admissibility and Accountability , 2005 .

[12]  Stephen D. Hart,et al.  The role of psychopathy in assessing risk for violence: Conceptual and methodological issues , 1998 .

[13]  H. Doll,et al.  Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24 827 people: systematic review and meta-analysis , 2012, BMJ : British Medical Journal.

[14]  S Lemeshow,et al.  The importance of assessing the fit of logistic regression models: a case study. , 1991, American journal of public health.

[15]  N Keiding,et al.  Individual survival time prediction using statistical models , 2005, Journal of Medical Ethics.

[16]  N. Wald,et al.  When can a risk factor be used as a worthwhile screening test? , 1999, BMJ.

[17]  Vernon L. Quinsey,et al.  Violent Recidivism of Mentally Disordered Offenders , 1993 .

[18]  Jennifer L. Skeem,et al.  Science Current Directions in Psychological Current Directions in Violence Risk Assessment on Behalf Of: Association for Psychological Science , 2022 .

[19]  P. Howard,et al.  Individual Confidence Intervals Do Not Inform Decision-Makers About the Accuracy of Risk Assessment Evaluations , 2010, Law and human behavior.

[20]  Paul S Appelbaum,et al.  Is it unethical to offer predictions of future violence? , 1992, Law and human behavior.

[21]  Grant T. Harris,et al.  Sixty-Six Years of Research on the Clinical Versus Actuarial Prediction of Violence , 2006 .

[22]  D. Cooke,et al.  Limitations of Diagnostic Precision and Predictive Utility in the Individual Case: A Challenge for Forensic Practice , 2009, Law and human behavior.