Which method predicts recidivism best?: a comparison of statistical, machine learning and data mining predictive models

Summary.  Using criminal population conviction histories of recent offenders, prediction mod els are developed that predict three types of criminal recidivism: general recidivism, violent recidivism and sexual recidivism. The research question is whether prediction techniques from modern statistics, data mining and machine learning provide an improvement in predictive performance over classical statistical methods, namely logistic regression and linear discrim inant analysis. These models are compared on a large selection of performance measures. Results indicate that classical methods do equally well as or better than their modern counterparts. The predictive performance of the various techniques differs only slightly for general and violent recidivism, whereas differences are larger for sexual recidivism. For the general and violent recidivism data we present the results of logistic regression and for sexual recidivism of linear discriminant analysis.

[1]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[3]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[4]  P. Robinson.  The interpretation of diagnostic tests. , 1987, Nuclear medicine communications.

[5]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[6]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[7]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[8]  D. Mossman.  Assessing predictions of violence: being accurate about accuracy. , 1994, Journal of consulting and clinical psychology.

[9]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[10]  Wilpen L. Gorr,et al.  Predicting criminal recidivism: A comparison of neural network models with statistical methods , 1996 .

[11]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[12]  D. Shapiro,et al.  The interpretation of diagnostic tests , 1999, Statistical methods in medical research.

[13]  M. Dolan,et al.  Violence risk prediction. Clinical and actuarial measures and the role of the Psychopathy Checklist. , 2000, The British journal of psychiatry : the journal of mental science.

[14]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[15]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[16]  J. Copas,et al.  The offender group reconviction scale: a statistical reconviction score for use by probation officers , 2002 .

[17]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[18]  Yvonne Vergouwe.  Validation of Clinical Prediction Models: Theory and Applications in Testicular Germ Cell Cancer , 2003 .

[19]  Mehryar Mohri,et al.  AUC Optimization vs. Error Rate Minimization , 2003, NIPS.

[20]  C. Ling,et al.  AUC: a Statistically Consistent and more Discriminating Measure than Accuracy , 2003, IJCAI.

[21]  D. Thornton,et al.  Distinguishing and Combining Risks for Sexual and Violent Recidivism , 2003, Annals of the New York Academy of Sciences.

[22]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[23]  Tom Fawcett,et al.  Robust Classification for Imprecise Environments , 2000, Machine Learning.

[24]  Wei-Yin Loh,et al.  A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms , 2000, Machine Learning.

[25]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[26]  Rich Caruana,et al.  Data mining in metric space: an empirical analysis of supervised learning performance criteria , 2004, ROCAI.

[27]  De Quick-Scan Reclassering; betrouwbaarheid en bruikbaarheid , 2004 .

[28]  Thomas Lengauer,et al.  ROCR: visualizing classifier performance in R , 2005, Bioinform..

[29]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[30]  David J. Hand,et al.  Mining Supervised Classification Performance Studies: A Meta-Analytic Investigation , 2008, J. Classif..

[31]  P. Howard,et al.  OGRS 3:the revised Offender Group Reconviction Scale , 2009 .

[32]  Stéphan Clémençon,et al.  Tree-Based Ranking Methods , 2009, IEEE Transactions on Information Theory.

[33]  Stefan Bogaerts,et al.  StatRec: inschatting van het recidivegevaar van verdachten van een misdrijf , 2009 .

[34]  Jeremy Coid,et al.  Applying Neural Networks and other statistical models to the classification of serious offenders and the prediction of recidivism , 2010 .

[35]  Yuan Y. Liu,et al.  A Comparison of Logistic Regression, Classification and Regression Tree, and Neural Networks Models in Predicting Violent Re-Offending , 2011 .

[36]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[37]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[38]  Max Kuhn,et al.  caret: Classification and Regression Training , 2015 .