How good is "evidence" from clinical studies of drug effects and why might such evidence fail in the prediction of the clinical utility of drugs?

Promising evidence from clinical studies of drug effects does not always translate to improvements in patient outcomes. In this review, we discuss why early evidence is often ill suited to the task of predicting the clinical utility of drugs. The current gap between initially described drug effects and their subsequent clinical utility results from deficits in the design, conduct, analysis, reporting, and synthesis of clinical studies-often creating conditions that generate favorable, but ultimately incorrect, conclusions regarding drug effects. There are potential solutions that could improve the relevance of clinical evidence in predicting the real-world effectiveness of drugs. What is needed is a new emphasis on clinical utility, with nonconflicted entities playing a greater role in the generation, synthesis, and interpretation of clinical evidence. Clinical studies should adopt strong design features, reflect clinical practice, and evaluate outcomes and comparisons that are meaningful to patients. Transformative changes to the research agenda may generate more meaningful and accurate evidence on drug effects to guide clinical decision making.

[1]  David Moher,et al.  SPIRIT 2013 Statement: defining standard protocol items for clinical trials. , 2015, Revista panamericana de salud publica = Pan American journal of public health.

[2]  Daniel Carpenter,et al.  Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA , 2014 .

[3]  John P A Ioannidis,et al.  More than a billion people taking statins?: Potential implications of the new cardiovascular guidelines. , 2014, JAMA.

[4]  John P A Ioannidis,et al.  Improving the drug development process: more not less randomized trials. , 2014, JAMA.

[5]  Thomas Agoritsas,et al.  How to use a subgroup analysis: users' guide to the medical literature. , 2014, JAMA.

[6]  Christopher H. Schmid,et al.  Characteristics of Networks of Interventions: A Description of a Database of 186 Published Networks , 2014, PloS one.

[7]  David Moher,et al.  Reducing waste from incomplete or unusable reports of biomedical research , 2014, The Lancet.

[8]  Harlan M Krumholz,et al.  Increasing value and reducing waste: addressing inaccessible research , 2014, The Lancet.

[9]  Iain Chalmers,et al.  How to increase value and reduce waste when research priorities are set , 2014, The Lancet.

[10]  R. Tibshirani,et al.  Increasing value and reducing waste in research design, conduct, and analysis , 2014, The Lancet.

[11]  U. Dirnagl,et al.  Biomedical research: increasing value, reducing waste , 2014, The Lancet.

[12]  Rustam Al-Shahi Salman,et al.  Increasing value and reducing waste in biomedical research regulation and management , 2014, The Lancet.

[13]  Lisa Hartling,et al.  Quality of reporting in systematic reviews of adverse events: systematic review , 2014, BMJ : British Medical Journal.

[14]  Nhan Tran,et al.  Republished research: Implementation research: what it is and how to do it , 2013, British Journal of Sports Medicine.

[15]  T. J. Moore,et al.  Development times, clinical testing, postmarket follow-up, and safety risks for the new drugs approved by the US food and drug administration: the class of 2008. , 2014, JAMA internal medicine.

[16]  S. Goodman,et al.  Raw data from clinical trials: within reach? , 2013, Trends in pharmacological sciences.

[17]  Omar Hasan,et al.  Doctors’ health: taking the lifecycle approach , 2013, BMJ.

[18]  J. Ioannidis,et al.  Biologic agents in rheumatology: unmet issues after 200 trials and $200 billion sales , 2013, Nature Reviews Rheumatology.

[19]  Frank Pétavy,et al.  Access to patient-level trial data--a boon to drug developers. , 2013, The New England journal of medicine.

[20]  Beate Wieseler,et al.  Completeness of Reporting of Patient-Relevant Clinical Trial Outcomes: Comparison of Unpublished Clinical Study Reports with Publicly Available Data , 2013, PLoS medicine.

[21]  J. Ioannidis,et al.  Comparative effectiveness of exercise and drug interventions on mortality outcomes: metaepidemiological study , 2015, British Journal of Sports Medicine.

[22]  John P A Ioannidis,et al.  Ensuring the integrity of clinical practice guidelines: a tool for protecting patients , 2013, BMJ : British Medical Journal.

[23]  Huseyin Naci,et al.  Assessing comparative effectiveness of new drugs before approval using prospective network meta-analyses. , 2013, Journal of clinical epidemiology.

[24]  Participant-level data and the new frontier in trial transparency. , 2013, The New England journal of medicine.

[25]  Tina Hernandez-Boussard,et al.  Overlapping meta-analyses on the same topic: survey of published studies , 2013, BMJ.

[26]  Nazmus Saquib,et al.  Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study , 2013, BMJ.

[27]  G. Alexander,et al.  The Food and Drug Administration Amendments Act and postmarketing commitments. , 2013, JAMA.

[28]  C. Gamble,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review , 2013, PloS one.

[29]  John P. A. Ioannidis,et al.  Undue industry influences that distort healthcare research, strategy, expenditure and practice: a review , 2013, European journal of clinical investigation.

[30]  I. Tannock,et al.  Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer. , 2013, Annals of oncology : official journal of the European Society for Medical Oncology.

[31]  J. Ioannidis,et al.  Bias in associations of emerging biomarkers with cardiovascular disease. , 2013, JAMA internal medicine.

[32]  Joel Lexchin,et al.  Surrogate outcomes in clinical trials: a cautionary tale. , 2013, JAMA internal medicine.

[33]  L. Bero Industry sponsorship and research outcome: a Cochrane review. , 2013, JAMA internal medicine.

[34]  J. Ioannidis,et al.  Most meta-analyses of drug interventions have narrow scopes and many focus on specific agents. , 2013, Journal of clinical epidemiology.

[35]  J. Ioannidis,et al.  Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes. , 2013, Journal of clinical epidemiology.

[36]  Tom Jefferson,et al.  Clinical study reports of randomised controlled trials: an exploratory review of previously confidential industry reports , 2013, BMJ Open.

[37]  J. Sterne,et al.  Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study , 2013, BMJ.

[38]  J. Ioannidis Mega-trials for blockbusters. , 2013, JAMA.

[39]  David Moher,et al.  Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane reviewa , 2012, Systematic Reviews.

[40]  T. Fleming,et al.  Biomarkers and surrogate endpoints in clinical trials , 2012, Statistics in medicine.

[41]  John P A Ioannidis,et al.  Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? , 2012, BMJ : British Medical Journal.

[42]  John P A Ioannidis,et al.  Empirical evaluation of very large treatment effects of medical interventions. , 2012, JAMA.

[43]  Ethan M Balk,et al.  Influence of Reported Study Design Characteristics on Intervention Effect Estimates From Randomized, Controlled Trials , 2012, Annals of Internal Medicine.

[44]  J. Lexchin,et al.  Pharmaceutical research and development: what do we get for all that money? , 2012, BMJ : British Medical Journal.

[45]  J. Cylus,et al.  Raising the bar for market authorisation of new drugs , 2012, BMJ : British Medical Journal.

[46]  S. Wyke,et al.  Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study , 2012, The Lancet.

[47]  S. Doggrell The ezetimibe controversy – can this be resolved by comparing the clinical trials with simvastatin and ezetimibe alone and together? , 2012, Expert opinion on pharmacotherapy.

[48]  E. Gale Post-marketing studies of new insulins: sales or science? , 2012, BMJ : British Medical Journal.

[49]  Mark A Hlatky,et al.  Editorial Repeat Revascularization Is a Faulty End Point for Clinical Trials Article See P 267 the Syntax Trial , 2022 .

[50]  Tom Jefferson,et al.  The Imperative to Share Clinical Study Reports: Recommendations from the Tamiflu Experience , 2012, PLoS medicine.

[51]  L. Trinquart,et al.  Underrepresentation of Elderly People in Randomised Controlled Trials. The Example of Trials of 4 Widely Prescribed Drugs , 2012, PloS one.

[52]  Gordon H Guyatt,et al.  Credibility of claims of subgroup effects in randomised controlled trials: systematic review , 2012, BMJ : British Medical Journal.

[53]  T. Jefferson,et al.  Rethinking credible evidence synthesis , 2012, BMJ : British Medical Journal.

[54]  A. Smyth,et al.  Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study , 2012, BMJ : British Medical Journal.

[55]  Lisa Bero,et al.  Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses , 2012, BMJ : British Medical Journal.

[56]  A. Hollis,et al.  Comment on “The economics of follow-on drug research and development: Trends in entry rates and the timing of development” , 2012, PharmacoEconomics.

[57]  V. Montori,et al.  The idolatry of the surrogate , 2011, BMJ : British Medical Journal.

[58]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[59]  Salomeh Keyhani,et al.  Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study , 2011, BMJ : British Medical Journal.

[60]  Vinay Prasad,et al.  The frequency of medical reversal. , 2011, Archives of internal medicine.

[61]  John P A Ioannidis,et al.  Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. , 2011, Journal of clinical epidemiology.

[62]  J. Ioannidis,et al.  Magnitude of effects in clinical trials published in high-impact general medical journals. , 2011, International journal of epidemiology.

[63]  J. Ioannidis,et al.  Public Availability of Published Research Data in High-Impact Journals , 2011, PloS one.

[64]  C. Furberg,et al.  Risk of serious adverse cardiovascular events associated with varenicline: a systematic review and meta-analysis , 2011, Canadian Medical Association Journal.

[65]  J. Cylus,et al.  Evidence of Comparative Efficacy Should Have a Formal Role in European Drug Approvals Article (published Version) (refereed) , 2022 .

[66]  Malcolm Rowland,et al.  Bridging the efficacy–effectiveness gap: a regulator's perspective on addressing variability of drug response , 2011, Nature Reviews Drug Discovery.

[67]  J. Ioannidis,et al.  Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. , 2011, JAMA.

[68]  P. Gøtzsche,et al.  Opening up data at the European Medicines Agency , 2011, BMJ : British Medical Journal.

[69]  Mary K. Kowal,et al.  Availability of comparative efficacy data at the time of drug approval in the United States. , 2011, JAMA.

[70]  Eric G Campbell,et al.  Conflicts of interest in cardiovascular clinical practice guidelines. , 2011, Archives of internal medicine.

[71]  Nicholas C. Ide,et al.  The ClinicalTrials.gov results database--update and key issues. , 2011, The New England journal of medicine.

[72]  Ian F Tannock,et al.  When are "positive" clinical trials in oncology truly positive? , 2011, Journal of the National Cancer Institute.

[73]  S. Goodman,et al.  A Systematic Examination of the Citation of Prior Research in Reports of Randomized, Controlled Trials , 2011, Annals of Internal Medicine.

[74]  D. Eyding,et al.  Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials , 2010, BMJ : British Medical Journal.

[75]  J. Ioannidis,et al.  The need to consider the wider agenda in systematic reviews and meta-analyses: breadth, timing, and depth of the evidence , 2010, BMJ : British Medical Journal.

[76]  D. Cohen Rosiglitazone: what went wrong? , 2010, BMJ : British Medical Journal.

[77]  Alexander J Sutton,et al.  Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials. , 2010, Journal of clinical epidemiology.

[78]  Douglas G Altman,et al.  Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. , 2010, JAMA.

[79]  R. Frank,et al.  The Rise and Fall of Gabapentin for Bipolar Disorder: A Case Study on Off-Label Pharmaceutical Diffusion , 2010, Medical care.

[80]  G. Guyatt,et al.  Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. , 2010, JAMA.

[81]  D. McCormick,et al.  Characteristics of published comparative effectiveness studies of medications. , 2010, JAMA.

[82]  Douglas G Altman,et al.  The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews , 2010, BMJ : British Medical Journal.

[83]  J. Ioannidis,et al.  Industry sponsorship and selection of comparators in randomized clinical trials , 2010, European journal of clinical investigation.

[84]  F. Song,et al.  Dissemination and publication of research findings: an updated review of related biases. , 2010, Health technology assessment.

[85]  E Ray Dorsey,et al.  Funding of US biomedical research, 2003-2008. , 2010, JAMA.

[86]  Kay Dickersin,et al.  Outcome reporting in industry-sponsored trials of gabapentin for off-label use. , 2009, The New England journal of medicine.

[87]  Philip W. Lavori,et al.  New, but not improved? Incorporating comparative-effectiveness information into FDA labeling. , 2009, The New England journal of medicine.

[88]  B. Munos Lessons from 60 years of pharmaceutical innovation , 2009, Nature Reviews Drug Discovery.

[89]  J. Ioannidis,et al.  The elderly were under-represented in osteoarthritis clinical trials. , 2009, Journal of clinical epidemiology.

[90]  John P A Ioannidis,et al.  Adverse events in randomized trials: neglected, restricted, distorted, and silenced. , 2009, Archives of internal medicine.

[91]  Isabelle Boutron,et al.  Reporting of safety results in published reports of randomized controlled trials. , 2009, Archives of internal medicine.

[92]  S. Paisley,et al.  Ezetimibe monotherapy for cholesterol lowering in 2,722 people: systematic review and meta‐analysis of randomized controlled trials , 2009, Journal of internal medicine.

[93]  R. Centor,et al.  Reassessment of clinical practice guidelines: go gently into that good night. , 2009, JAMA.

[94]  R. Califf,et al.  Scientific evidence underlying the ACC/AHA clinical practice guidelines. , 2009, JAMA.

[95]  E. Douglass The ADVANTAGE Seeding Trial: A Review of Internal Documents , 2009 .

[96]  P. Glasziou,et al.  Avoidable waste in the production and reporting of evidence , 2009 .

[97]  Lisa Bero,et al.  Reporting Bias in Drug Trials Submitted to the Food and Drug Administration: Review of Publication and Presentation , 2008, PLoS medicine.

[98]  H. Leufkens,et al.  Choice of Comparator in Active Control Trials of New Drugs , 2008, The Annals of pharmacotherapy.

[99]  P. Rochon,et al.  Published randomized controlled trials of drug therapy for dementia often lack complete data on harm. , 2008, Journal of clinical epidemiology.

[100]  Allen D. Roses,et al.  Pharmacogenetics in drug discovery and development: a translational perspective , 2008, Nature Reviews Drug Discovery.

[101]  Peter C Gøtzsche,et al.  Industry-supported meta-analyses compared with meta-analyses with non-profit or no support : Differences in methodological quality and conclusions , 2015 .

[102]  John P A Ioannidis,et al.  Perfect study, poor evidence: interpretation of biases preceding study design. , 2008, Seminars in hematology.

[103]  Carl Heneghan,et al.  What is missing from descriptions of treatment in trials and reviews? , 2008, BMJ : British Medical Journal.

[104]  Sally Hopewell,et al.  Reporting of adverse events in systematic reviews can be improved: survey results. , 2008, Journal of clinical epidemiology.

[105]  S. Nissen ENHANCE and ACCORD: Controversy over surrogate end points , 2008, Current cardiology reports.

[106]  S. Sismondo How Pharmaceutical Industry Funding Affects Trial Outcomes: Causal Structures and Responses , 2008, Social science & medicine.

[107]  Allen J. Taylor,et al.  Does ENHANCE diminish confidence in lowering LDL or in ezetimibe? , 2008, The New England journal of medicine.

[108]  John Ioannidis,et al.  Exploring the Geometry of Treatment Networks , 2008, Annals of Internal Medicine.

[109]  Douglas G Altman,et al.  Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study , 2008, BMJ : British Medical Journal.

[110]  Sergio Sismondo,et al.  Pharmaceutical company funding and its consequences: a qualitative systematic review. , 2008, Contemporary clinical trials.

[111]  Matthias Briel,et al.  Early stopping of randomized clinical trials for overt efficacy is problematic. , 2008, Journal of clinical epidemiology.

[112]  Blair T. Johnson,et al.  Initial Severity and Antidepressant Benefits: A Meta-Analysis of Data Submitted to the Food and Drug Administration , 2008, PLoS medicine.

[113]  B. Thiers,et al.  Eligibility Criteria of Randomized Controlled Trials Published in High-Impact General Medical Journals: A Systematic Sampling Review , 2008 .

[114]  J. Ioannidis,et al.  Persistence of contradicted claims in the literature. , 2007, JAMA.

[115]  Veronica Yank,et al.  Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study , 2007, BMJ : British Medical Journal.

[116]  Allen F. Shaughnessy,et al.  Clinical Epidemiology: A Basic Science for Clinical Medicine , 2007, BMJ : British Medical Journal.

[117]  Peter C Gøtzsche,et al.  Data extraction errors in meta-analyses that use standardized mean differences. , 2007, JAMA.

[118]  Gordon H Guyatt,et al.  Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials , 2007, BMJ : British Medical Journal.

[119]  H. Leufkens,et al.  Availability of comparative trials for the assessment of new medicines in the European Union at the moment of market authorization. , 2007, British journal of clinical pharmacology.

[120]  H. Yazici,et al.  A survey of inclusion of the time element when reporting adverse effects in randomised controlled trials of cyclo-oxygenase-2 and tumour necrosis factor α inhibitors , 2006, Annals of the rheumatic diseases.

[121]  Peter C Gøtzsche,et al.  Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review , 2006, BMJ : British Medical Journal.

[122]  M. Chren,et al.  Narrative Review: The Promotion of Gabapentin: An Analysis of Internal Industry Documents , 2006, Annals of Internal Medicine.

[123]  J. Ioannidis,et al.  Origin and funding of the most frequently cited papers in medicine: database analysis , 2006, BMJ : British Medical Journal.

[124]  L. Gluud Bias in clinical intervention research. , 2006, American journal of epidemiology.

[125]  R. Matthews,et al.  What are the implications of optimism bias in clinical research? , 2006, The Lancet.

[126]  Gordon H Guyatt,et al.  Randomized trials stopped early for benefit: a systematic review. , 2005, JAMA.

[127]  H. Rehman Under-representation of the elderly in clinical trials. , 2005, European journal of internal medicine.

[128]  J. Ioannidis Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[129]  Nicola J Cooper,et al.  The use of systematic reviews when designing studies , 2005, Clinical trials.

[130]  Thomas A Trikalinos,et al.  Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. , 2005, Journal of clinical epidemiology.

[131]  J. Ioannidis,et al.  Relative Citation Impact of Various Study Designs in the Health Sciences , 2005, JAMA.

[132]  Richard Smith,et al.  Medical Journals Are an Extension of the Marketing Arm of Pharmaceutical Companies , 2005, PLoS medicine.

[133]  Peter C Gøtzsche,et al.  [Better reporting of harms in randomized trials: an extension of the CONSORT statement]. , 2005, Ugeskrift for laeger.

[134]  F. Rockhold The Society for Clinical Trials supports United States legislation mandating trials registration , 2005, Clinical trials.

[135]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2005, Circulation.

[136]  D. Altman,et al.  Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors , 2005, BMJ : British Medical Journal.

[137]  R. Stafford,et al.  National trends in cyclooxygenase-2 inhibitor use since market release: nonselective diffusion of a selectively cost-effective innovation. , 2005, Archives of internal medicine.

[138]  P. Rothwell Subgroup analysis in randomised controlled trials: importance, indications, and interpretation , 2005, The Lancet.

[139]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2004, JAMA.

[140]  Thomas R Fleming,et al.  Surrogate endpoints and FDA's accelerated approval process. , 2005, Health affairs.

[141]  M. Egger,et al.  Risk of cardiovascular events and rofecoxib: cumulative meta-analysis , 2004, The Lancet.

[142]  Eric J Topol,et al.  Failing the public health--rofecoxib, Merck, and the FDA. , 2004, The New England journal of medicine.

[143]  J. Ioannidis,et al.  Safety reporting in randomized trials of mental health interventions. , 2004, The American journal of psychiatry.

[144]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[145]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2004, JAMA.

[146]  Andrew D Oxman,et al.  HARLOT plc: an amalgamation of the world's two oldest professions , 2003, BMJ : British Medical Journal.

[147]  D. Stryer,et al.  Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. , 2003, JAMA.

[148]  R. Glasgow,et al.  Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. , 2003, American journal of public health.

[149]  B. Djulbegovic,et al.  Pharmaceutical industry sponsorship and research outcome and quality: systematic review , 2003, BMJ : British Medical Journal.

[150]  C. Gross,et al.  Scope and impact of financial conflicts of interest in biomedical research: a systematic review. , 2003, JAMA.

[151]  Harlan M Krumholz,et al.  Representation of the elderly, women, and minorities in heart failure clinical trials. , 2002, Archives of internal medicine.

[152]  Ethan M Balk,et al.  Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. , 2002, JAMA.

[153]  C. Gluud,et al.  Citation bias of hepato-biliary randomized clinical trials. , 2002, Journal of clinical epidemiology.

[154]  Christian Gluud,et al.  Reported Methodologic Quality and Discrepancies between Large and Small Randomized Trials in Meta-Analyses , 2001, Annals of Internal Medicine.

[155]  E. Ernst,et al.  Assessment of therapeutic safety in systematic reviews: literature review , 2001, BMJ : British Medical Journal.

[156]  J. Ioannidis,et al.  Comparison of evidence of treatment effects in randomized and nonrandomized studies. , 2001, JAMA.

[157]  K. Alexander,et al.  Representation of elderly persons and women in published randomized trials of acute coronary syndromes. , 2001, JAMA.

[158]  J. Ioannidis,et al.  Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. , 2001, JAMA.

[159]  S. Assmann,et al.  Subgroup analysis and other (mis)uses of baseline data in clinical trials , 2000, The Lancet.

[160]  Gordon H. Guyatt,et al.  Guides to the Medical Literature XXV . Evidence-Based Medicine : Principles for Applying the Users ’ Guides to Patient Care , 2000 .

[161]  C. Coltman,et al.  Underrepresentation of patients 65 years of age or older in cancer-treatment trials. , 1999, The New England journal of medicine.

[162]  M. Mayo-Smith,et al.  Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. , 1999, JAMA.

[163]  J. Grimshaw,et al.  Potential benefits, limitations, and harms of clinical guidelines , 1999, BMJ.

[164]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[165]  J. Ioannidis Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. , 1998, JAMA.

[166]  D J Torgerson,et al.  Pragmatic trials: lab meets bedside , 2019, The British journal of dermatology.

[167]  S Garattini,et al.  Are me-too drugs justified? , 1997, Journal of nephrology.

[168]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[169]  D. DeMets,et al.  Surrogate End Points in Clinical Trials: Are We Being Misled? , 1996, Annals of Internal Medicine.

[170]  J. Grimshaw,et al.  Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations , 1993, The Lancet.

[171]  I Chalmers,et al.  Underreporting research is scientific misconduct. , 1990, JAMA.

[172]  P. Gøtzsche Reference bias in reports of drug trials. , 1987, British medical journal.