Introducing EMMIE: an evidence rating scale to encourage mixed-method crime prevention synthesis reviews

ObjectivesThis paper describes the need for, and the development of, a coding system to distil the quality and coverage of systematic reviews of the evidence relating to crime prevention interventions. The starting point for the coding system concerns the evidence needs of policymakers and practitioners.MethodsThe proposed coding scheme (EMMIE) builds on previous scales that have been developed to assess the probity, coverage and utility of evidence both in health and criminal justice. It also draws on the principles of realist synthesis and review.ResultsThe proposed EMMIE scale identifies five dimensions to which systematic reviews intended to inform crime prevention should speak. These are the Effect of intervention, the identification of the causal Mechanism(s) through which interventions are intended to work, the factors that Moderate their impact, the articulation of practical Implementation issues, and the Economic costs of intervention.ConclusionsSystematic reviews of crime prevention, and the primary studies on which they are based, typically address the question of effect size, but are often silent on the other dimensions of EMMIE. This lacuna of knowledge is unhelpful to practitioners who want to know more than what might work to reduce crime. The EMMIE framework is intended to encourage the collection of primary data regarding these issues and the synthesis of such knowledge in future systematic reviews.

[1]  Dennis P. Rosenbaum,et al.  Community crime prevention: A review and synthesis of the literature , 1988 .

[2]  Ulrike Dapp,et al.  Development, feasibility and performance of a health risk appraisal questionnaire for older persons , 2007, BMC medical research methodology.

[3]  Mark W. Lipsey,et al.  The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. , 1993, The American psychologist.

[4]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[5]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[6]  Shane D. Johnson,et al.  Cost-benefit analysis for crime science: making cost-benefit analysis useful through a portfolio of outcomes , 2005 .

[7]  P Mills,et al.  The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration , 2008, Quality & Safety in Health Care.

[8]  C. McDougall,et al.  Benefit-Cost Analyses of Sentencing: A Systematic Review , 2008 .

[9]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[10]  C. McDougall,et al.  Benefit-cost analyses of sentencing , 2008 .

[11]  Larry V. Hedges,et al.  Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model , 1996 .

[12]  Anthony A. Braga,et al.  The Effects of Focused Deterrence Strategies on Crime , 2012 .

[13]  Ray Pawson,et al.  RAMESES publication standards: realist syntheses , 2013, BMC Medicine.

[14]  F. Hughes,et al.  Quality of evidence , 2013, BDJ.

[15]  J. Wilson,et al.  BROKEN WINDOWS: THE POLICE AND NEIGHBOURHOOD SAFETY , 1982 .

[16]  王林,et al.  CONSORT , 2011 .

[17]  Demonstration, Exemplification, Duplication and Replication in Evaluation Research , 1996 .

[18]  Catherine Hewitt,et al.  Are criminologists describing randomized controlled trials in ways that allow us to assess them? Findings from a sample of crime and justice trials , 2010 .

[19]  Marilyn J. Haring,et al.  Rigor in Data Synthesis: A Case Study of Reliability in Meta-analysis , 1982 .

[20]  David Weisburd,et al.  Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: challenging folklore in evaluation research in crime and justice , 2010 .

[21]  Hannah R Rothstein,et al.  A basic introduction to fixed‐effect and random‐effects models for meta‐analysis , 2010, Research synthesis methods.

[22]  N. Tilley,et al.  Further improving reporting in crime and justice: an addendum to Perry, Weisburd and Hewitt (2010) , 2012 .

[23]  Allen I. Huffcutt,et al.  Development of a new outlier statistic for meta-analytic data. , 1995 .

[24]  Shane D. Johnson,et al.  Implementing Information Science in Policing: Mapping the Evidence Base , 2014 .

[25]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[26]  Kanxing Zhao,et al.  Plasminogen activator inhibitor-1 4G/5G polymorphism and retinopathy risk in type 2 diabetes: a meta-analysis , 2013, BMC Medicine.

[27]  H. Williams,et al.  How well are randomized controlled trials reported in the dermatology literature? , 2000, Archives of dermatology.

[28]  D. Moher,et al.  CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials , 2010, BMJ : British Medical Journal.

[29]  Joshua C. Hinkle,et al.  Understanding the Mechanisms Underlying Broken Windows Policing , 2015 .

[30]  Gordon H Guyatt,et al.  GrADe : what is “ quality of evidence ” and why is it important to clinicians ? rATING quALITY of evIDeNCe AND STreNGTH of reCommeNDATIoNS , 2022 .

[31]  Ray Pawson,et al.  Evidence-based Policy: The Promise of `Realist Synthesis' , 2002 .

[32]  David Moher,et al.  Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews , 2007, BMC medical research methodology.

[33]  Ray Pawson,et al.  Evidence-based policy: a realist perspective. , 2006 .

[34]  Charlotte Gill.  Missing links: how descriptive validity impacts the policy relevance of randomized controlled trials in criminology , 2011 .

[35]  Peter Reuter,et al.  Preventing Crime: What Works, What Doesn't, What's Promising. Research in Brief. National Institute of Justice. , 1998 .

[36]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[37]  Shane D. Johnson,et al.  INSTALLING ALLEY-GATES: PRACTICAL LESSONS FROM BURGLARY PREVENTION PROJECTS , 2001 .

[38]  Shane D. Johnson,et al.  Crime displacement: what we know, what we don’t know, and what it means for crime reduction , 2014 .

[39]  F. Bryant,et al.  Methodological issues in the meta‐analysis of quasi‐experiments , 1984 .

[40]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[41]  D. Moher,et al.  CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials , 2010, Journal of pharmacology & pharmacotherapeutics.

[42]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies , 2007, The Lancet.

[43]  Matthias Egger,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies , 2007, PLoS medicine.

[44]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.