Meta-analysis of data from animal studies: A practical guide

Meta-analyses of data from human studies are invaluable resources in the life sciences and the methods to conduct these are well documented. Similarly there are a number of benefits in conducting meta-analyses on data from animal studies; they can be used to inform clinical trial design, or to try and explain discrepancies between preclinical and clinical trial results. However there are inherit differences between animal and human studies and so applying the same techniques for the meta-analysis of preclinical data is not straightforward. For example preclinical studies are frequently small and there is often substantial heterogeneity between studies. This may have an impact on both the method of calculating an effect size and the method of pooling data. Here we describe a practical guide for the meta-analysis of data from animal studies including methods used to explore sources of heterogeneity.

[1]  M. Pittler.  Systematic Reviews in Health Care: Meta‐analysis in Context , 2010 .

[2]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[3]  D. Howells,et al.  Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy , 2010, PLoS biology.

[4]  L. Mignini,et al.  Methodological quality of systematic reviews of animal studies: a survey of reviews of basic research , 2006, BMC medical research methodology.

[5]  Steffen Mickenautsch,et al.  Systematic reviews, systematic error and the acquisition of clinical knowledge , 2010, BMC medical research methodology.

[6]  J. Durlak.  How to select, calculate, and interpret effect sizes. , 2009, Journal of pediatric psychology.

[7]  Robert E. Slavin,et al.  Best-Evidence Synthesis: Why Less Is More , 1987 .

[8]  B. Sibbald,et al.  Understanding controlled trials: Why are randomised controlled trials important? , 1998 .

[9]  I. Cuthill,et al.  Reporting : The ARRIVE Guidelines for Reporting Animal Research , 2010 .

[10]  Amit X Garg,et al.  Systematic review and meta-analysis: when one study is just not enough. , 2008, Clinical journal of the American Society of Nephrology : CJASN.

[11]  G. Donnan,et al.  Hypothermia in animal models of acute ischaemic stroke: a systematic review and meta-analysis. , 2007, Brain : a journal of neurology.

[12]  G. Donnan,et al.  1,026 Experimental treatments in acute stroke , 2006, Annals of neurology.

[13]  David W Howells,et al.  Factors Affecting the Apparent Efficacy and Safety of Tissue Plasminogen Activator in Thrombotic Occlusion Models of Stroke: Systematic Review and Meta-Analysis , 2010, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[14]  M. Macleod,et al.  Dopamine agonists in animal models of Parkinson's disease: a systematic review and meta-analysis. , 2011, Parkinsonism & related disorders.

[15]  Elizabeth Tipton,et al.  Erratum: Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[16]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[17]  U. Dirnagl,et al.  Evidence for the Efficacy of NXY-059 in Experimental Focal Cerebral Ischaemia Is Confounded by Study Quality , 2008, Stroke.

[18]  J. Grimshaw,et al.  Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments , 2013, PLoS medicine.

[19]  Michael Nilsson,et al.  A Systematic Review and Meta-Analysis of Erythropoietin in Experimental Stroke , 2010, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[20]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[21]  Malcolm Macleod,et al.  How can we improve the pre-clinical development of drugs for stroke? , 2007, Trends in Neurosciences.

[22]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[23]  T. Baguley.  Standardized or simple effect size: what should be reported? , 2009, British journal of psychology.

[24]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[25]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[26]  Alex J. Sutton,et al.  A Systematic Review of Systematic Reviews and Meta-Analyses of Animal Experiments with Guidelines for Reporting , 2006 .

[27]  Joseph Beyene,et al.  The ratio of means method as an alternative to mean differences for analyzing continuous outcome variables in meta-analysis: A simulation study , 2008 .

[28]  U. Dirnagl,et al.  Empirical Evidence of Bias in the Design of Experimental Stroke Studies: A Metaepidemiologic Approach , 2008, Stroke.

[29]  S. Lazic,et al.  A call for transparent reporting to optimize the predictive value of preclinical research , 2012, Nature.

[30]  C. Lengeler,et al.  Language bias in randomised controlled trials published in English and German , 1997, The Lancet.

[31]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1997, Statistics in medicine.

[32]  D. Cook,et al.  Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions , 1997, Annals of Internal Medicine.

[33]  Alex J. Sutton,et al.  Modelling publication bias in meta-analysis: a review , 2000 .

[34]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[35]  Carlijn R Hooijmans,et al.  A step-by-step guide to systematically identify all relevant animal studies , 2012, Laboratory animals.

[36]  Emily S. Sena,et al.  Developing our understanding of the pathogenesis of Alzheimer's disease using a systematically identified dataset of interventions tested in transgenic mouse models , 2012 .

[37]  D. Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. , 2001, Journal of the American Podiatric Medical Association.

[38]  L. Hedges,et al.  The Handbook of Research Synthesis and Meta-Analysis , 2009 .

[39]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[40]  Nathalie Percie du Sert,et al.  Systematic review and meta-analysis of the self-administration of opioids in rats and nonhuman primates to provide evidence for the choice of species in models of abuse potential , 2013 .

[41]  M. Macleod,et al.  Improving the translational hit of experimental treatments in multiple sclerosis , 2010, Multiple sclerosis.

[42]  Wim Van den Noortgate,et al.  Three-level meta-analysis of dependent effect sizes , 2012, Behavior Research Methods.

[43]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[44]  J. Ioannidis.  Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[45]  Stefan Michiels,et al.  Meta-analysis when only the median survival times are known: A comparison with individual patient data results , 2005, International Journal of Technology Assessment in Health Care.

[46]  Michele Tarsilla.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[47]  T C Chalmers,et al.  Bias in treatment assignment in controlled clinical trials. , 1983, The New England journal of medicine.

[48]  Peter Sandercock,et al.  Systematic Review and Meta-Analysis of the Efficacy of Tirilazad in Experimental Stroke , 2007, Stroke.

[49]  J. Ioannidis,et al.  Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases , 2013, PLoS biology.

[50]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[51]  M. Macleod,et al.  Treatment of intracerebral hemorrhage in animal models: Meta‐analysis , 2011, Annals of neurology.

[52]  W. G. Cochran.  The combination of estimates from different experiments. , 1954 .

[53]  Frederic M. Lord,et al.  On the Statistical Treatment of Football Numbers. , 1953 .

[54]  I. Olkin,et al.  Improving the quality of reporting of randomized controlled trials. The CONSORT statement. , 1996, JAMA.

[55]  D. Howells,et al.  Can Animal Models of Disease Reliably Inform Human Studies? , 2010, PLoS medicine.

[56]  Trisha Greenhalgh,et al.  How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses) , 1997 .

[57]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[58]  R. Simes,et al.  Confronting publication bias: a cohort design for meta-analysis. , 1987, Statistics in medicine.

[59]  I. Cuthill,et al.  Effect size, confidence interval and statistical significance: a practical guide for biologists , 2007, Biological reviews of the Cambridge Philosophical Society.