Evolution and Translation of Research Findings: From Bench to Where?

The credibility and replication of research findings evolve over time, as data accumulate. However, translation of postulated research promises to real-life biomedical applications is uncommon. In some fields of research, we may observe diminishing effects for the strength of research findings and rapid alternations of exaggerated claims and extreme contradictions--the "Proteus Phenomenon." While these phenomena are probably more prominent in the basic sciences, similar manifestations have been documented even in clinical trials and they may undermine the credibility of clinical research. Significance-chasing bias may be in part responsible, but the greatest threat may come from the poor relevance and scientific rationale and thus low pre-study odds of success of research efforts. Given that we currently have too many research findings, often with low credibility, replication and rigorous evaluation become as important as or even more important than discovery. Credibility, replication, and translation are all desirable properties of research findings, but are only modestly correlated. In this essay, I discuss some of the evidence (or lack thereof) for the process of evolution and translation of research findings, with emphasis on the biomedical sciences.

[1]  J. Ioannidis,et al.  Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[2]  L. Stewart,et al.  Time to publication for results of clinical trials. , 2007, The Cochrane database of systematic reviews.

[3]  D G Altman,et al.  Indirect comparisons of competing interventions. , 2005, Health technology assessment.

[4]  D. Lawlor,et al.  Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? , 2004, The Lancet.

[5]  J. Attia,et al.  Haplotype analysis of VDR gene polymorphisms: a meta-analysis , 2004, Osteoporosis International.

[6]  A. Hofman,et al.  Promoter and 3'-untranslated-region haplotypes in the vitamin d receptor gene predispose to osteoporotic fracture: the rotterdam study. , 2005, American journal of human genetics.

[7]  Susan R. Johnson,et al.  Effects of Estrogen or Estrogen/ Progestin Regimens on Heart Disease Risk Factors in Postmenopausal Women: The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial , 1995 .

[8]  M. Jennions,et al.  Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution , 2002, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[9]  Mike Clarke,et al.  Discussion sections in reports of controlled trials published in general medical journals. , 2002, JAMA.

[10]  J. Gray Evidence-Based Healthcare , 1997 .

[11]  T C Chalmers,et al.  Changes in clinical trials mandated by the advent of meta-analysis. , 1996, Statistics in medicine.

[12]  F. Marincola Translational Medicine: A two-way road , 2003, Journal of Translational Medicine.

[13]  G. Donnan,et al.  Systematic Review and Metaanalysis of the Efficacy of FK506 in Experimental Stroke , 2005, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[14]  J. H. Comroe,et al.  Scientific basis for the support of biomedical science. , 1976, Science.

[15]  Matthew B Schabath,et al.  An Evolutionary Perspective on Single-Nucleotide Polymorphism Screening in Molecular Cancer Epidemiology , 2004, Cancer Research.

[16]  D. Altman,et al.  Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors , 2005, BMJ : British Medical Journal.

[17]  John P A Ioannidis,et al.  Translation of highly promising basic science research into clinical applications. , 2003, The American journal of medicine.

[18]  Paul R. Rosenbaum,et al.  Replicating Effects and Biases , 2001 .

[19]  J. Lau,et al.  The impact of high-risk patients on the results of clinical trials. , 1997, Journal of clinical epidemiology.

[20]  H S Sacks,et al.  The relationship between study design, results, and reporting of randomized clinical trials of HIV infection. , 1997, Controlled clinical trials.

[21]  The Clinical Interpretation of Research , 2005, PLoS medicine.

[22]  Ján,et al.  Effects of estrogen or estrogen/progestin regimens on heart disease risk factors in postmenopausal women. The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial. The Writing Group for the PEPI Trial. , 1995, JAMA.

[23]  P. Jais How frequent is altered gene expression among susceptibility genes to human complex disorders? , 2005, Genetics in Medicine.

[24]  B Djulbegovic,et al.  Acceptable regret in medical decision making. , 1999, Medical hypotheses.

[25]  Manfred Hauben,et al.  Data mining in pharmacovigilance: lessons from phantom ships , 2006, European Journal of Clinical Pharmacology.

[26]  N. Freemantle,et al.  Selective serotonin reuptake inhibitors versus tricyclic and heterocyclic antidepressants: comparison of drug adherence. , 2000, The Cochrane database of systematic reviews.

[27]  Franz Porzsolt,et al.  The fading of reported effectiveness. A meta-analysis of randomised controlled trials , 2006, BMC medical research methodology.

[28]  Douglas G Altman,et al.  Epidemiology and reporting of randomised trials published in PubMed journals , 2005, The Lancet.

[29]  J. Marshall,et al.  PRECLINICAL MODELS OF SHOCK AND SEPSIS: WHAT CAN THEY TELL US? , 2005, Shock.

[30]  L. Gluud Bias in clinical intervention research. , 2006, American journal of epidemiology.

[31]  J. Ioannidis Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[32]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[33]  H. Straatman,et al.  Evaluation of treatment regimens to cure Helicobacter pylori infection—a meta‐analysis , 1999, Alimentary pharmacology & therapeutics.

[34]  George Davey Smith,et al.  Meta-analysis of randomised controlled trials , 1997, The Lancet.

[35]  Eric J Topol,et al.  Failing the public health--rofecoxib, Merck, and the FDA. , 2004, The New England journal of medicine.

[36]  Rachel Churchill,et al.  Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time. , 2004, Journal of clinical epidemiology.

[37]  R. Matthews,et al.  What are the implications of optimism bias in clinical research? , 2006, The Lancet.

[38]  M. Brown,et al.  A Randomised Controlled Trial of Vitamin E in Patients with Coronary Disease: The Cambridge Heart Antioxidant Study (CHAOS) , 1996 .

[39]  G. Donnan,et al.  1,026 Experimental treatments in acute stroke , 2006, Annals of neurology.

[40]  D. Altman,et al.  Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research , 2004, Canadian Medical Association Journal.

[41]  Kennon Heard,et al.  Emergency medicine animal research: does use of randomization and blinding affect the results? , 2003, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[42]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[43]  A. Uitterlinden,et al.  The Association between Common Vitamin D Receptor Gene Variations and Osteoporosis: A Participant-Level Meta-Analysis , 2006, Annals of Internal Medicine.

[44]  Thomas A Trikalinos,et al.  Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. , 2005, Journal of clinical epidemiology.

[45]  David W Howells,et al.  Pooling of Animal Experimental Data Reveals Influence of Study Design and Publication Bias , 2004, Stroke.

[46]  Melissa S. Anderson,et al.  Normal Misbehavior: Scientists Talk about the Ethics of Research , 2006, Journal of empirical research on human research ethics : JERHRE.

[47]  Richard Horton,et al.  Surgical research or comic opera: questions, but few answers , 1996, The Lancet.

[48]  F. Porzsolt,et al.  “Fading of Reported Effectiveness” Bias: Longitudinal Meta-Analysis of Randomized Controlled Trials , 2006 .

[49]  Stefan Leucht,et al.  Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. , 2006, The American journal of psychiatry.

[50]  J. Ioannidis,et al.  Concordance of functional in vitro data and epidemiological associations in complex disease genetics , 2006, Genetics in Medicine.

[51]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[52]  J. Ioannidis,et al.  Replication validity of genetic association studies , 2001, Nature Genetics.

[53]  J. Ioannidis Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. , 1998, JAMA.

[54]  J. Ioannidis,et al.  Randomised trials comparing chemotherapy regimens for advanced non-small cell lung cancer: biases and evolution over time. , 2003, European journal of cancer.

[55]  J. Eisman,et al.  Prediction of bone density from vitamin D receptor alleles , 1994, Nature.

[56]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2005, Circulation.

[57]  Richard Smith,et al.  Patient Safety Requires a New Way to Publish Clinical Trials , 2006, PLoS clinical trials.

[58]  R Smith,et al.  Comroe and Dripps revisited. , 1987, British medical journal.

[59]  P. Rothwell,et al.  Factors That Can Affect the External Validity of Randomised Controlled Trials , 2006, PLoS clinical trials.

[60]  R. Recker,et al.  The Association of Bone Mineral Density with Vitamin D Receptor Gene Polymorphisms , 1999, Osteoporosis International.