Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment

Actuarial risk assessments might be unduly perceived as a neutral way to counteract implicit bias and increase the fairness of decisions made at almost every juncture of the criminal justice system, from pretrial release to sentencing, parole and probation. In recent times these assessments have come under increased scrutiny, as critics claim that the statistical techniques underlying them might reproduce existing patterns of discrimination and historical biases that are reflected in the data. Much of this debate is centered around competing notions of fairness and predictive accuracy, resting on the contested use of variables that act as "proxies" for characteristics legally protected against discrimination, such as race and gender. We argue that a core ethical debate surrounding the use of regression in risk assessments is not simply one of bias or accuracy. Rather, it's one of purpose. If machine learning is operationalized merely in the service of predicting individual future crime, then it becomes difficult to break cycles of criminalization that are driven by the iatrogenic effects of the criminal justice system itself. We posit that machine learning should not be used for prediction, but rather to surface covariates that are fed into a causal model for understanding the social, structural and psychological drivers of crime. We propose an alternative application of machine learning and causal inference away from predicting risk scores to risk mitigation.

[1]  B. Harcourt,et al.  Risk as a Proxy for Race , 2010 .

[2]  Kelly Hannah-Moffat,et al.  Assembling Risk and the Restructuring of Penal Control , 2006 .

[3]  Christopher T. Lowenkamp,et al.  False Positives, False Negatives, and False Analyses: A Rejoinder to "Machine Bias: There's Software Used across the Country to Predict Future Criminals. and It's Biased against Blacks" , 2016 .

[4]  Jennifer L. Skeem,et al.  Risk Assessment in Criminal Sentencing. , 2016, Annual review of clinical psychology.

[5]  Kelly Hannah-Moffat,et al.  The Uncertainties of Risk Assessment: Partiality, Transparency, and Just Decisions , 2015 .

[6]  Jure Leskovec,et al.  Human Decisions and Machine Predictions , 2017, The quarterly journal of economics.

[7]  Christopher T. Lowenkamp,et al.  Gender, risk assessment, and sanctioning: The cost of treating women like men. , 2016, Law and human behavior.

[8]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[9]  D. A. Andrews,et al.  The Recent Past and Near Future of Risk and/or Need Assessment , 2006 .

[10]  J. Kleinberg,et al.  Prediction Policy Problems. , 2015, The American economic review.

[11]  R. Kitchin,et al.  Big Data, new epistemologies and paradigm shifts , 2014, Big Data Soc..

[12]  Cynthia Lum,et al.  Why do evaluation researchers in crime and justice choose non-experimental methods? , 2005 .

[13]  Joanne Belknap,et al.  The Invisible Woman: Gender, Crime, and Justice , 1995 .

[14]  David Weisburd,et al.  Randomized Experiments in Criminal Justice Policy: Prospects and Problems , 2000 .

[15]  Meda Chesney‐Lind,et al.  Girls' Crime and Woman's Place: Toward a Feminist Model of Female Delinquency , 1989 .

[16]  David P. Farrington,et al.  Evidence-based Crime Prevention: The Effectiveness of CCTV , 2004 .

[17]  Grant Duwe,et al.  Sacrificing Accuracy for Transparency in Recidivism Risk Assessment: The Impact of Classification Method on Predictive Performance , 2016 .

[18]  Mark Hardy,et al.  Practitioner perspectives on risk: Using governmentality to understand contemporary probation practice , 2014 .

[19]  Traci Schlesinger,et al.  The Failure of Race Neutral Policies: How Mandatory Terms and Sentencing Enhancements Contribute to Mass Racialized Incarceration , 2011 .

[20]  M. Stevenson Distortion of Justice: How the Inability to Pay Bail Affects Case Outcomes , 2018, The Journal of Law, Economics, and Organization.

[21]  Joshua C. Cochran,et al.  Examining Prison Effects on Recidivism: A Regression Discontinuity Approach , 2017 .

[22]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[23]  D. Greiner,et al.  Randomized Control Trials in the United States Legal Profession , 2016 .

[24]  Sally S. Simpson,et al.  FEMINIST THEORY, CRIME, AND JUSTICE* , 1989 .

[25]  D. A. Andrews,et al.  Classification for Effective Rehabilitation , 1990 .

[26]  Antonio Andrés Pueyo,et al.  THE PSYCHOLOGY OF CRIMINAL CONDUCT , 2008 .

[27]  Nolan G. Pope,et al.  The Unintended Impact of Pretrial Detention on Case Outcomes: Evidence from New York City Arraignments , 2017, The Journal of Law and Economics.

[28]  C. Blumberg Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction , 2016 .

[29]  R. Defina,et al.  For incapacitation, there is no time like the present: The lagged effects of prisoner reentry on property and violent crime rates , 2010 .

[30]  L. Sherman,et al.  Low-intensity community supervision for low-risk offenders: a randomized, controlled trial , 2010 .

[31]  Francis T. Cullen,et al.  From Nothing Works to What Works: Changing Professional Ideology in the 21st Century , 2001 .

[32]  Kelly Hannah-Moffat,et al.  Criminogenic needs and the transformative risk subject , 2005 .

[33]  Stephen A. Fennell,et al.  Due Process at Sentencing: An Empirical and Legal Analysis of the Disclosure of Presentence Reports in Federal Courts , 1980 .

[34]  Sarah Desmarais Jay Singh,et al.  Risk Assessment Instruments Validated and Implemented in Correctional Settings in the United States , 2013 .

[35]  D. Greiner Causal Inference in Civil Rights Litigation , 2008 .

[36]  H Roberts,et al.  Risk Society: Towards a New Modernity , 1994 .

[37]  Melissa Hamilton Back to the Future: The Influence of Criminal History on Risk Assessment , 2015 .

[38]  Arpita Gupta,et al.  The Heavy Costs of High Bail: Evidence from Judge Randomization , 2016, The Journal of Legal Studies.

[39]  Kelly Hannah-Moffat,et al.  Sacrosanct or Flawed: Risk, Accountability and Gender- Responsive Penal Politics , 2010 .

[40]  Margareth Etienne Legal and Practical Implications of Evidence-Based Sentencing by Judges , 2009 .

[41]  M. C. Elish,et al.  Situating methods in the magic of Big Data and AI , 2018 .

[42]  Kristy Holtfreter,et al.  Assessing Recidivism Risk Across Female Pathways to Crime , 2006 .

[43]  Nicole Gonzalez Van Cleve,et al.  Criminal Justice Through “Colorblind” Lenses: A Call to Examine the Mutual Constitution of Race and Criminal Justice , 2015, Law & Social Inquiry.

[44]  Franklin E. Zimring,et al.  The Great American Crime Decline , 2006 .

[45]  Paulo J. G. Lisboa,et al.  Making machine learning models interpretable , 2012, ESANN.

[46]  Elizabeth Levy Paluck,et al.  The Promising Integration of Qualitative Methods and Field Experiments , 2010 .

[47]  Cynthia Rudin,et al.  Interpretable classification models for recidivism prediction , 2015, 1503.07810.

[48]  Daniel S. Nagin,et al.  Prisons Do Not Reduce Recidivism , 2011 .

[49]  Kelly Hannah-Moffat Actuarial Sentencing: An “Unsettled” Proposition , 2013 .

[50]  D. Rubin Matched Sampling for Causal Effects , 2006 .

[51]  Andrew D. Selbst Disparate Impact in Big Data Policing , 2017 .

[52]  Jonathan Simon,et al.  The New Penology: Notes on the Emerging Strategy of Corrections and Its Implications , 1992 .

[53]  Andrew D. Selbst,et al.  Big Data's Disparate Impact , 2016 .

[54]  S. Prins,et al.  Can we avoid reductionism in risk reduction? , 2018, Theoretical criminology.

[55]  Crystal S. Yang,et al.  The Effects of Pre-Trial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges , 2016 .

[56]  Sonja B. Starr Evidence-Based Sentencing and the Scientific Rationalization of Discrimination , 2013 .

[57]  M. Kearns,et al.  Fairness in Criminal Justice Risk Assessments: The State of the Art , 2017, Sociological Methods & Research.

[58]  Daniela M. Witten,et al.  An Introduction to Statistical Learning: with Applications in R , 2013 .