Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments

Background The vast majority of medical interventions introduced into clinical development prove unsafe or ineffective. One prominent explanation for the dismal success rate is flawed preclinical research. We conducted a systematic review of preclinical research guidelines and organized recommendations according to the type of validity threat (internal, construct, or external) or programmatic research activity they primarily address. Methods and Findings We searched MEDLINE, Google Scholar, Google, and the EQUATOR Network website for all preclinical guideline documents published up to April 9, 2013 that addressed the design and conduct of in vivo animal experiments aimed at supporting clinical translation. To be eligible, documents had to provide guidance on the design or execution of preclinical animal experiments and represent the aggregated consensus of four or more investigators. Data from included guidelines were independently extracted by two individuals for discrete recommendations on the design and implementation of preclinical efficacy studies. These recommendations were then organized according to the type of validity threat they addressed. A total of 2,029 citations were identified through our search strategy. From these, we identified 26 guidelines that met our eligibility criteria—most of which were directed at neurological or cerebrovascular drug development. Together, these guidelines offered 55 different recommendations. Some of the most common recommendations included performance of a power calculation to determine sample size, randomized treatment allocation, and characterization of disease phenotype in the animal model prior to experimentation. Conclusions By identifying the most recurrent recommendations among preclinical guidelines, we provide a starting point for developing preclinical guidelines in other disease domains. We also provide a basis for the study and evaluation of preclinical research practice. Please see later in the article for the Editors' Summary

[1]  J. Noseworthy,et al.  The impact of blinding on the results of a randomized, placebo‐controlled multiple sclerosis clinical trial , 1994, Neurology.

[2]  A D Oxman,et al.  Randomisation to protect against selection bias in healthcare trials. , 2011, The Cochrane database of systematic reviews.

[3]  J. Montaner,et al.  Recommendations guide for experimental animal models in stroke research. , 2011, Neurologia.

[4]  Andrew S.C. Rice,et al.  Animal models and the prediction of efficacy in clinical trials of analgesic drugs: A critical appraisal and call for uniform reporting standards , 2008, PAIN.

[5]  R. Virmani,et al.  Drug-Eluting Stents in Preclinical Studies: Updated Consensus Recommendations for Preclinical Evaluation , 2008, Circulation. Cardiovascular interventions.

[6]  L. Ferguson,et al.  The 10 basic requirements for a scientific paper reporting antioxidant, antimutagenic or anticarcinogenic potential of test substances in in vitro experiments and animal studies in vivo. , 2003, Food and chemical toxicology : an international journal published for the British Industrial Biological Research Association.

[7]  N. Pullen,et al.  The translational challenge in the development of new and effective therapies for endometriosis: a review of confidence from published preclinical efficacy studies. , 2011, Human reproduction update.

[8]  C. Begley,et al.  Drug development: Raise standards for preclinical cancer research , 2012, Nature.

[9]  Ulrich Dirnagl,et al.  Reprint: Good Laboratory Practice: Preventing Introduction of Bias at the Bench , 2009, Stroke.

[10]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.

[11]  M. Macleod,et al.  Dopamine agonists in animal models of Parkinson's disease: a systematic review and meta-analysis. , 2011, Parkinsonism & related disorders.

[12]  X. Montalban,et al.  [Guidelines on the appropriate use of animal models for developing therapies in multiple sclerosis]. , 2012, Revista de neurología (Ed. impresa).

[13]  U. Dirnagl,et al.  Empirical Evidence of Bias in the Design of Experimental Stroke Studies: A Metaepidemiologic Approach , 2008, Stroke.

[14]  S. Greenfield,et al.  Clinical practice guidelines we can trust , 2011 .

[15]  M. Emborg,et al.  Beyond Access vs. Protection in Trials of Innovative Therapies , 2010, Science.

[16]  G. Donnan,et al.  1,026 Experimental treatments in acute stroke , 2006, Annals of neurology.

[17]  R. Campbell,et al.  The Lambeth Conventions: guidelines for the study of arrhythmias in ischaemia infarction, and reperfusion. , 1988, Cardiovascular research.

[18]  R. Bolli,et al.  Myocardial Protection at a Crossroads: The Need for Translation Into Clinical Therapy , 2004, Circulation research.

[19]  H. Winn,et al.  RODENT STROKE MODEL GUIDELINES FOR PRECLINICAL STROKE TRIALS (1ST EDITION). , 2009, Journal of experimental stroke & translational medicine.

[20]  I. Cuthill,et al.  Survey of the Quality of Experimental Design, Statistical Analysis and Reporting of Research Using Animals , 2009, PloS one.

[21]  M. Stolina,et al.  Rodent Preclinical Models for Developing Novel Antiarthritic Molecules: Comparative Biology and Preferred Methods for Evaluating Efficacy , 2010, Journal of biomedicine & biotechnology.

[22]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[23]  Marc Fisher,et al.  Update of the Stroke Therapy Academic Industry Roundtable Preclinical Recommendations , 2009, Stroke.

[24]  M. Cases,et al.  Inroads to Predict in Vivo Toxicology—An Introduction to the eTOX Project , 2012, International journal of molecular sciences.

[25]  Nigel W. Hardy,et al.  Promoting coherent minimum reporting guidelines for biological and biomedical investigations: the MIBBI project , 2008, Nature Biotechnology.

[26]  M. Benatar,et al.  Enhancing translation: Guidelines for standard pre-clinical experiments in mdx mice , 2012, Neuromuscular Disorders.

[27]  J. Montaner,et al.  Recommendations guide for experimental animal models in stroke research , 2011 .

[28]  V. Steele,et al.  Guidance for development of chemopreventive agents , 1994, Journal of cellular biochemistry. Supplement.

[29]  D. Redelmeier,et al.  Translation of research evidence from animals to humans. , 2006, JAMA.

[30]  Michael Chopp,et al.  Stem Cell Therapy as an Emerging Paradigm for Stroke (STEPS) II , 2011, Stroke.

[31]  L. Cronbach,et al.  Designing evaluations of educational and social programs , 1983 .

[32]  B. Reynolds Stem Cell Therapies as an Emerging Paradigm in Stroke (STEPS): Bridging Basic and Clinical Science for Cellular and Neurogenic Factor Therapy in Treating Stroke , 2009, Stroke.

[33]  Kimberly Scearce-Levie,et al.  Animal Models of Alzheimer's Disease: Modeling Targets, Not Disease Accelerating Drug Discovery for Alzheimer's Disease: Best Practices for Preclinical Animal Studies , 2022 .

[34]  M. Ashburner,et al.  The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration , 2007, Nature Biotechnology.

[35]  Pedro Ramos-Cabrer,et al.  Recommendations guide for experimental animal models in stroke research. , 2011, Neurologia.

[36]  J. Menegazzi,et al.  Drug administration in animal studies of cardiac arrest does not reflect human clinical experience. , 2007, Resuscitation.

[37]  M. Brennan,et al.  New live mycobacterial vaccines: the Geneva consensus on essential steps towards clinical development. , 2005, Vaccine.

[38]  B. Moreno,et al.  Guía de uso adecuado de modelos animales para el desarrollo de terapias en esclerosis múltiple , 2012 .

[39]  P. Bath,et al.  Effects of NXY‐059 in experimental stroke: an individual animal meta‐analysis , 2009, British journal of pharmacology.

[40]  G. Guyatt,et al.  Systematic review of reviews including animal studies addressing therapeutic interventions for sepsis* , 2010, Critical care medicine.

[41]  Janet Woodcock,et al.  The FDA critical path initiative and its influence on new drug development. , 2008, Annual review of medicine.

[42]  I. Cuthill,et al.  Reporting : The ARRIVE Guidelines for Reporting Animal Research , 2010 .

[43]  Albert C. Ludolph,et al.  Guidelines for preclinical animal research in ALS/MND: A consensus meeting , 2010, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases.

[44]  Susan Margulies,et al.  Combination therapies for traumatic brain injury: prospective considerations. , 2009, Journal of neurotrauma.

[45]  D. Howells,et al.  Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy , 2010, PLoS biology.

[46]  Rodney C. Samaco,et al.  Preclinical research in Rett syndrome: setting the foundation for translational success , 2012, Disease Models & Mechanisms.

[47]  Michelle E. Kho,et al.  AGREE II: advancing guideline development, reporting and evaluation in health care , 2010, Canadian Medical Association Journal.

[48]  M. Grounds,et al.  Towards developing standard operating procedures for pre-clinical testing in the mdx mouse model of Duchenne muscular dystrophy , 2008, Neurobiology of Disease.

[49]  D. Howells,et al.  Can Animal Models of Disease Reliably Inform Human Studies? , 2010, PLoS medicine.

[50]  S. Lazic,et al.  A call for transparent reporting to optimize the predictive value of preclinical research , 2012, Nature.

[51]  Iveta Simera,et al.  EQUATOR: reporting guidelines for health research , 2008, Open medicine : a peer-reviewed, independent, open-access journal.

[52]  J. E. Kranz,et al.  Design, power, and interpretation of studies in the standard murine model of ALS , 2008, Amyotrophic lateral sclerosis : official publication of the World Federation of Neurology Research Group on Motor Neuron Diseases.

[53]  Alex J Sutton,et al.  A Systematic Review of Systematic Reviews and Meta-Analyses of Animal Experiments with Guidelines for Reporting , 2006, Journal of environmental science and health. Part. B, Pesticides, food contaminants, and agricultural wastes.

[54]  Stroke Therapy Academic Industry Roundtable Recommendations for standards regarding preclinical neuroprotective and restorative drug development. , 1999, Stroke.

[55]  I. Cuthill,et al.  Animal Research: Reporting In Vivo Experiments: The ARRIVE Guidelines , 2010, British journal of pharmacology.

[56]  R. Bellomo,et al.  Acute renal failure – definition, outcome measures, animal models, fluid therapy and information technology needs: the Second International Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group , 2004, Critical care.

[57]  J. Grimshaw,et al.  North of England evidence based guidelines development project: methods of guideline development , 1996, BMJ.

[58]  W J Sibbald,et al.  Introducing Critical Appraisal to studies of animal models investigating novel therapies in sepsis. , 1996, Critical care medicine.

[59]  John P A Ioannidis,et al.  Translation of highly promising basic science research into clinical applications. , 2003, The American journal of medicine.

[60]  I. Kola,et al.  Can the pharmaceutical industry reduce attrition rates? , 2004, Nature Reviews Drug Discovery.

[61]  P. Lambiase,et al.  The Lambeth Conventions (II): guidelines for the study of animal and human ventricular and supraventricular arrhythmias. , 2013, Pharmacology & therapeutics.

[62]  Douglas G Altman,et al.  Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study , 2008, BMJ : British Medical Journal.

[63]  James A. Anderson,et al.  Should preclinical studies be registered? , 2012, Nature Biotechnology.