Machine Learning Risk Assessments in Criminal Justice Settings

This chapter provides a general introduction to forecasting criminal behavior in criminal justice settings. A common application is to predict at a parole hearing whether the inmate being considered for release is a significant risk to public safety. It may surprise some that criminal justice forecasts of risk have been used by decision-makers in the United States since at least the 1920s. Over time, statistical methods have replaced clinical methods, leading to improvements in forecasting accuracy. The gains were at best gradual until recently, when the increasing availability of very large datasets, powerful computers, and new statistical procedures began to produce dramatic improvements. But, criminal justice forecasts of risk are inextricably linked to criminal justice decision-making and to both the legitimate and illegitimate interests of various stakeholders. Sometimes, criticisms of risk assessment become convenient vehicles to raise broader issues around social inequality. There are, in short, always political considerations, ethical complexities, and judgement calls for which there can be no technical fix. The recent controversy about “racial bias” in risk instruments is a salient example. 1.1 Some Introductory Caveats There are claims, increasingly common, that one can sensibly apply machine learning without understanding the underlying fundamentals. Not surprisingly, one hears this a lot from business firms trying to sell “data analytics” software. At the other extreme, are claims from academics in computer science and statistics that unless a data analyst is well-versed in those underlying fundamentals, that analyst will know just enough to be dangerous. Complicating these debates is that there is a great deal about machine learning that is not well understood by any academic discipline. In many instances, practices has evolved much faster than the requisite theory. An effort is made in the pages ahead to strike a balance between the two extremes. The danger is that both camps will be dissatisfied. And that is OK. There is an old saying that you know when you are doing something right if everyone is mad at you. © Springer Nature Switzerland AG 2019 R. Berk, Machine Learning Risk Assessments in Criminal Justice Settings, https://doi.org/10.1007/978-3-030-02272-3_1 1

[1]  Trevor J. Hastie,et al.  Confidence intervals for random forests: the jackknife and the infinitesimal jackknife , 2013, J. Mach. Learn. Res..

[2]  L. Ohlin,et al.  A Comparison of Alternative Methods of Parole Prediction , 1952 .

[3]  A. Sen,et al.  Collective Choice and Social Welfare , 2017 .

[4]  Sonja B. Starr Evidence-Based Sentencing and the Scientific Rationalization of Discrimination , 2013 .

[5]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[6]  J. Messing,et al.  The Lethality Screen , 2017, Journal of interpersonal violence.

[7]  R. Real,et al.  AUC: a misleading measure of the performance of predictive distribution models , 2008 .

[8]  Carter C. Price,et al.  Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations , 2013 .

[9]  David J. Hand,et al.  Measuring classifier performance: a coherent alternative to the area under the ROC curve , 2009, Machine Learning.

[10]  D. Webster,et al.  Intimate Partner Violence Risk Assessment Validation Study: (515662006-001) , 2005 .

[11]  M. Tonry Legal and Ethical Issues in the Prediction of Recidivism , 2014 .

[12]  Leslie T. Wilkins Problems with Existing Prediction Studies and Future Research Needs , 1980 .

[13]  Greg Ridgeway,et al.  Generalized Boosted Models: A guide to the gbm package , 2006 .

[14]  Arun K. Kuchibhotla,et al.  A Model Free Perspective for Linear Regression: Uniform-in-model Bounds for Post Selection Inference , 2018 .

[15]  B. M. Pötscher,et al.  MODEL SELECTION AND INFERENCE: FACTS AND FICTION , 2005, Econometric Theory.

[16]  J. Monahan,et al.  Judicial Decision Thresholds for Violence Risk Management , 2003 .

[17]  J. Monahan,et al.  Science Current Directions in Psychological Current Directions in Violence Risk Assessment on Behalf Of: Association for Psychological Science , 2022 .

[18]  Alex Sánchez A Tutorial Review of Microarray Data Analysis , 2008 .

[19]  Andrew von Hirsch,et al.  Censure and sanctions , 1994 .

[20]  David Robinson,et al.  The detection of criminal groups in real-world fused data: using the graph-mining algorithm “GraphExtract” , 2018, Security Informatics.

[21]  Seth Neel,et al.  An Empirical Study of Rich Subgroup Fairness for Machine Learning , 2018, FAT.

[22]  David Mease,et al.  Boosted Classification Trees and Class Probability/Quantile Estimation , 2007, J. Mach. Learn. Res..

[23]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[24]  Fred L. Cheesman,et al.  Using Risk Assessment to Inform Sentencing Decisions for Nonviolent Offenders in Virginia , 2007 .

[25]  H. Leeb,et al.  CAN ONE ESTIMATE THE UNCONDITIONAL DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS? , 2003, Econometric Theory.

[26]  J. Monahan Predicting Violent Behavior: An Assessment of Clinical Techniques , 1981 .

[27]  J. Carbonell,et al.  MMPI-2 with male and female state and federal prison inmates. , 1999 .

[28]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[29]  Jacquelyn C. Campbell,et al.  The Oklahoma Lethality Assessment Study: A Quasi-Experimental Evaluation of the Lethality Assessment Program , 2015, Social Service Review.

[30]  O. D. Duncan,et al.  The Efficiency of Prediction in Criminology , 1949, American Journal of Sociology.

[31]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[32]  Seth Neel,et al.  Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness , 2017, ICML.

[33]  Bo Chen,et al.  MnasNet: Platform-Aware Neural Architecture Search for Mobile , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Donald E. Knuth The art of computer programming: fundamental algorithms , 1969 .

[35]  Charles Petzold,et al.  The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine , 2008 .

[36]  A. Reiss, The Accuracy, Efficiency, and Validity of a Prediction Instrument , 1951, American Journal of Sociology.

[37]  S. D. Gottfredson,et al.  Statistical Risk Assessment: Old Problems and New Applications , 2006 .

[38]  J. Kagan,et al.  Rational choice in an uncertain world , 1988 .

[39]  Cynthia Rudin,et al.  Interpretable classification models for recidivism prediction , 2015, 1503.07810.