Comparative Effectiveness Research : Role of Observational Investigations 991

Introduction: Comparative effectiveness research (CER) seeks to inform clinical decisions between alternate treatment strategies using data that reflects real patient populations and real-world clinical scenarios for the purpose of improving patient outcomes. There are multiple clinical situations where the unique characteristics of observational investigations can inform medical decision-making within the CER paradigm. Accordingly, it is critical for clinicians to appreciate the strengths and limitations of observational research, particularly as they apply to CER. Methods: This review focuses on the role of observational research in CER. We discuss the concept of evidence hierarchies as they relate to observational research and CER, review the scope and nature of observational research, present the rationale for its inclusion in CER investigations, discuss potential sources of bias in observational investigations as well as strategies used to compensate for these biases, and discuss a framework to implement observational research in CER. Conclusions: The CER paradigm recognizes the limitations of hierarchical models of evidence and favors application of a strength-of-evidence model. In this model, observational research fills gaps in randomized clinical trial data and is particularly valuable to investigate effectiveness, harms, prognosis, and infrequent outcomes as well as in circumstances where randomization is not possible and in studies of many surgical populations. Observational investigations must be designed with careful consideration of potential sources of bias and must incorporate strategies to control such bias prospectively, and their results must be reported in a uniform and transparent fashion. When these conditions can be achieved, observational research represents a valuable and critical component of modern CER.

[1]  B. Barrett,et al.  Bridging the gap between conventional and alternative medicine. , 2000, The Journal of family practice.

[2]  K. Starko,et al.  Reye's syndrome and salicylate use. , 1980, Pediatrics.

[3]  R J Lilford,et al.  Equipoise and the ethics of randomization. , 1995, Journal of the Royal Society of Medicine.

[4]  T. Hugh.  Surgery and the randomised controlled trial: past, present and future , 1999, The Medical journal of Australia.

[5]  B. Maron,et al.  Prevalence of sudden cardiac death during competitive sports activities in Minnesota high school athletes. , 1998, Journal of the American College of Cardiology.

[6]  J. Higgins,et al.  Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. , 2007, International journal of epidemiology.

[7]  Peter Cummings,et al.  The relative merits of risk ratios and odds ratios. , 2009, Archives of pediatrics & adolescent medicine.

[8]  J. Ioannidis,et al.  Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. , 2001, JAMA.

[9]  G. D'aliberti,et al.  Myelomeningocele: long-term neurosurgical treatment and follow-up in 202 patients. , 2007, Journal of neurosurgery.

[10]  C. Viscoli,et al.  Developing improved observational methods for evaluating therapeutic effectiveness. , 1990, The American journal of medicine.

[11]  A R Feinstein,et al.  Clinical epidemiological quality in molecular genetic research: the need for methodological standards. , 1999, JAMA.

[12]  Peter Jüni,et al.  Quality of reporting of randomized trials as a measure of methodologic quality. , 2002, JAMA.

[13]  G. Lewith,et al.  Complementary and alternative medicine — with a difference , 2004, The Medical journal of Australia.

[14]  Arif Khan,et al.  Placebo in mood disorders: the tail that wags the dog , 2003 .

[15]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Preventive medicine.

[16]  I. Olkin,et al.  Meta-analysis of observational studies in epidemiology - A proposal for reporting , 2000 .

[17]  G. Lewith,et al.  Why do people seek treatment by alternative medicine? , 1985, British medical journal.

[18]  L. Hartling,et al.  Challenges in Systematic Reviews of Therapeutic Devices and Procedures , 2005, Annals of Internal Medicine.

[19]  Irving Kirsch,et al.  Response expectancy as a determinant of experience and behavior. , 1985 .

[20]  David Moher,et al.  Assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care Program , 2008 .

[21]  D Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. , 2001, Journal of the American Podiatric Medical Association.

[22]  V. McCormack,et al.  Issues in the reporting of epidemiological studies: a survey of recent practice , 2004, BMJ : British Medical Journal.

[23]  M. Hotopf,et al.  Bias in psychiatric case–control studies , 2007, British Journal of Psychiatry.

[24]  C Zoccali,et al.  The valuable contribution of observational studies to nephrology. , 2007, Kidney international.

[25]  David Atkins,et al.  Challenges in Using Nonrandomized Studies in Systematic Reviews of Treatment Interventions , 2005, Annals of Internal Medicine.

[26]  N. Pearce,et al.  What does the odds ratio estimate in a case-control study? , 1993, International journal of epidemiology.

[27]  Jan P Vandenbroucke,et al.  Observational Research, Randomised Trials, and Two Views of Medical Science , 2008, PLoS medicine.

[28]  R. Moore,et al.  Reporting of adverse effects in clinical trials should be improved: lessons from acute postoperative pain. , 1999, Journal of pain and symptom management.

[29]  Douglas G Altman,et al.  Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses , 2003, BMJ : British Medical Journal.

[30]  Sander Greenland,et al.  Theoretical Epidemiology: Principles of Occurrence Research in Medicine , 1986 .

[31]  S. Greenfield,et al.  Comparative Effectiveness Research: A Report From the Institute of Medicine , 2009, Annals of Internal Medicine.

[32]  J. Meakins.  Innovation in surgery: the rules of evidence. , 2002, American journal of surgery.

[33]  P. Rothwell,et al.  External validity of randomised controlled trials: “To whom do the results of this trial apply?” , 2005, The Lancet.

[34]  J. Lantos,et al.  The ethics of randomization , 1991, CA: a cancer journal for clinicians.

[35]  Bill Abrams,et al.  The observational research handbook : understanding how consumers live with your product , 2000 .

[36]  M. Egger,et al.  Risk of cardiovascular events and rofecoxib: cumulative meta-analysis , 2004, The Lancet.

[37]  I. Olkin,et al.  Improving the quality of reports of meta‐analyses of randomised controlled trials: the QUOROM statement , 2000, Revista espanola de salud publica.

[38]  C. Mulrow,et al.  Current methods of the US Preventive Services Task Force: a review of the process. , 2001, American journal of preventive medicine.

[39]  R. Chou,et al.  Challenges in Systematic Reviews That Assess Treatment Harms , 2005, Annals of Internal Medicine.

[40]  N. Dreyer,et al.  Registries for Evaluating Patient Outcomes: A User’s Guide , 2010 .

[41]  Sheena Derry,et al.  BMC Clinical Pharmacology BioMed Central BMC 1 2001, Clinical Pharmacology , 2001 .

[42]  Xiaonan Xue,et al.  Estimating the relative risk in cohort studies and clinical trials of common outcomes. , 2003, American journal of epidemiology.

[43]  N. Roos,et al.  Use of claims data systems to evaluate health care outcomes. Mortality and reoperation following prostatectomy. , 1987, JAMA.

[44]  W. Wittmann,et al.  Evaluating complementary medicine : lessons to be learned from evaluation research , 2002 .

[45]  G. Lewith,et al.  Circular instead of hierarchical: methodological principles for the evaluation of complex interventions , 2006 .

[46]  R. Waldman,et al.  Aspirin as a risk factor in Reye's syndrome. , 1982, JAMA.

[47]  S. Retsas.  Treatment at random: the ultimate science or the betrayal of Hippocrates? , 2004, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[48]  Prospects Liubolong.  Improving quality. , 1990, The Psychiatric clinics of North America.

[49]  M. Barry,et al.  Patient-reported complications and follow-up treatment after radical prostatectomy. The National Medicare Experience: 1988-1990 (updated June 1993). , 1993, Urology.

[50]  Peter Hernon,et al.  The Sage Encyclopedia of Social Science Research Methods: Edited by Michael S. Lewis-Beck, Alan Bryman, and Tim Futing Liao. 3 vols. Thousand Oaks, CA: Sage Publications, 2004. 1305 pp. $450.00. ISBN 0-7619-2363-2 , 2004 .

[51]  A. Hartz,et al.  A comparison of observational studies and randomized, controlled trials. , 2000, The New England journal of medicine.

[52]  J. Ray.  Evidence in upheaval: incorporating observational data into clinical practice. , 2002, Archives of internal medicine.

[53]  I. Olkin,et al.  Improving the quality of reporting of randomized controlled trials. The CONSORT statement. , 1996, JAMA.

[54]  J. Concato.  Observationalversus experimental studies: What’s the evidence for a hierarchy? , 2004, NeuroRX.

[55]  C. Weel,et al.  Systematic reviews incorporating evidence from nonrandomized study designs: reasons for caution when estimating health effects , 2005, European Journal of Clinical Nutrition.

[56]  M. Bigby.  Challenges to the hierarchy of evidence: does the emperor have no clothes? , 2001, Archives of dermatology.

[57]  E C Hill,et al.  Clear cell carcinoma of the cervix and vagina in young women. A report of six cases with association of maternal stilbestrol therapy and adenosis of the vagina. , 1973, American journal of obstetrics and gynecology.

[58]  Richard Birtwhistle,et al.  Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. , 2003, BMC medical research methodology.

[59]  C Bain,et al.  Interpreting the evidence: choosing between randomised and non-randomised studies , 1999, BMJ.

[60]  Paul Glasziou,et al.  Assessing the quality of research , 2004, BMJ : British Medical Journal.

[61]  Douglas K Owens,et al.  Grading the Strength of a Body of Evidence When Comparing Medical Interventions , 2009 .

[62]  E. Robinson,et al.  Information presentation and decisions to enter clinical trials: a hypothetical trial of hormone replacement therapy. , 2000, Social science & medicine.

[63]  T A Sheldon,et al.  A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. , 2000, Health technology assessment.

[64]  T. Cook,et al.  Lessons Learned about Evaluation in the United States and Some Possible Implications for Europe , 1998 .

[65]  R Day,et al.  Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. VIGOR Study Group. , 2000, The New England journal of medicine.

[66]  G. Guyatt,et al.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations , 2008, BMJ : British Medical Journal.

[67]  Ian Shrier,et al.  Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. , 2007, American journal of epidemiology.

[68]  J. Kleijnen,et al.  Placebo effect in double-blind clinical trials: a review of interactions with medications , 1994, The Lancet.

[69]  J. Vandenbroucke,et al.  Observational research and evidence-based medicine: What should we teach young physicians? , 1998, Journal of clinical epidemiology.

[70]  J. Ioannidis,et al.  Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement , 2004, Annals of Internal Medicine.

[71]  R. N. Stauffer.  Ten-year follow-up study of total hip replacement. , 1982, The Journal of bone and joint surgery. American volume.

[72]  J. Concato,et al.  Randomized, controlled trials, observational studies, and the hierarchy of research designs. , 2000, The New England journal of medicine.

[73]  A. Herbst,et al.  Adenocarcinoma of the vagina. Association of maternal stilbestrol therapy with tumor appearance in young women. , 1971, The New England journal of medicine.

[74]  N McKoy,et al.  Systems to rate the strength of scientific evidence. , 2002, Evidence report/technology assessment.

[75]  R. Horwitz,et al.  Complexity and contradiction in clinical trial research. , 1987, The American journal of medicine.

[76]  B. Ettinger,et al.  Clinic visits and hospital admissions for care of acid-related upper gastrointestinal disorders in women using alendronate for osteoporosis. , 1998, The American journal of managed care.

[77]  J. Avorn.  In defense of pharmacoepidemiology--embracing the yin and yang of drug research. , 2007, The New England journal of medicine.

[78]  N. Black.  Why we need observational studies to evaluate the effectiveness of health care , 1996, BMJ.

[79]  B. Stricker,et al.  Confounding by indication: an example of variation in the use of epidemiologic terminology. , 1999, American journal of epidemiology.

[80]  T C Chalmers,et al.  Randomized versus historical controls for clinical trials. , 1982, The American journal of medicine.

[81]  Roger Chou,et al.  Initial highly-active antiretroviral therapy with a protease inhibitor versus a non-nucleoside reverse transcriptase inhibitor: discrepancies between direct and indirect meta-analyses , 2006, The Lancet.

[82]  R. McLeod,et al.  Should we be performing more randomized controlled trials evaluating surgical operations? , 1995, Surgery.

[83]  M. Koster,et al.  The History of DES, Lessons to be Learned , 2005, Pharmacy World and Science.

[84]  A. Dobson,et al.  Quality of reporting of observational longitudinal research. , 2005, American journal of epidemiology.

[85]  J. Vandenbroucke.  Benefits and harms of drug treatments , 2004, BMJ : British Medical Journal.

[86]  S. Shoor,et al.  Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and non-selective non-steroidal anti-inflammatory drugs: nested case-control study , 2005, The Lancet.

[87]  C. Bombardier,et al.  Methodological quality and homogeneity influenced agreement between randomized trials and nonrandomized studies of the same intervention for back pain. , 2008, Journal of clinical epidemiology.

[88]  S. Patten.  Selection bias in studies of major depression using clinical subjects. , 2000, Journal of clinical epidemiology.

[89]  Jan P Vandenbroucke,et al.  When are observational studies as credible as randomised trials? , 2004, The Lancet.

[90]  J. Ioannidis,et al.  Safety reporting in randomized trials of mental health interventions. , 2004, The American journal of psychiatry.