Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retr

OBJECTIVES The goal of comparative effectiveness analysis is to examine the relationship between two variables, treatment, or exposure and effectiveness or outcome. Unlike data obtained through randomized controlled trials, researchers face greater challenges with causal inference with observational studies. Recognizing these challenges, a task force was formed to develop a guidance document on methodological approaches to addresses these biases. METHODS The task force was commissioned and a Chair was selected by the International Society for Pharmacoeconomics and Outcomes Research Board of Directors in October 2007. This report, the second of three reported in this issue of the Journal, discusses the inherent biases when using secondary data sources for comparative effectiveness analysis and provides methodological recommendations to help mitigate these biases. RESULTS The task force report provides recommendations and tools for researchers to mitigate threats to validity from bias and confounding in measurement of exposure and outcome. Recommendations on design of study included: the need for data analysis plan with causal diagrams; detailed attention to classification bias in definition of exposure and clinical outcome; careful and appropriate use of restriction; extreme care to identify and control for confounding factors, including time-dependent confounding. CONCLUSIONS Design of nonrandomized studies of comparative effectiveness face several daunting issues, including measurement of exposure and outcome challenged by misclassification and confounding. Use of causal diagrams and restriction are two techniques that can improve the theoretical basis for analyzing treatment effects in study populations of more homogeneity, with reduced loss of generalizability.

[1]  James M. Robins,et al.  Causal diagrams for epidemiologic research. , 1999 .

[2]  J. Robins,et al.  Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. , 2000, Epidemiology.

[3]  A. Pablos-Mendez,et al.  Run-in periods in randomized trials: implications for the application of results in clinical practice. , 1998, JAMA.

[4]  S. Erickson,et al.  Compliance from Self-Reported versus Pharmacy Claims Data with Metered-Dose Inhalers , 2001, The Annals of pharmacotherapy.

[5]  S. Schneeweiss,et al.  Developments in Post‐marketing Comparative Effectiveness Research , 2007, Clinical pharmacology and therapeutics.

[6]  P. Rothwell.  Subgroup analysis in randomised controlled trials: importance, indications, and interpretation , 2005, The Lancet.

[7]  An analysis of the exclusion criteria used in observational pharmacoepidemiological studies , 2007 .

[8]  B H McFarland,et al.  The validity of medicaid pharmacy claims for estimating drug use among elderly nursing home residents: The Oregon experience. , 2000, Journal of clinical epidemiology.

[9]  B. Motheral,et al.  The use of claims databases for outcomes research: rationale, challenges, and strategies. , 1997, Clinical therapeutics.

[10]  Zhigang Duan,et al.  Limits of observational data in determining outcomes from cancer therapy , 2008, Cancer.

[11]  Sebastian Schneeweiss,et al.  Accuracy of Medicare claims-based diagnosis of acute myocardial infarction: estimating positive predictive value on the basis of review of hospital records. , 2004, American heart journal.

[12]  Peter M. Rothwell,et al.  Reliable assessment of the effects of treatments on mortality and major morbidity , 2007 .

[13]  David Atkins,et al.  Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part I. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[14]  W. Ray,et al.  Evaluating medication effects outside of clinical trials: new-user designs. , 2003, American journal of epidemiology.

[15]  A Wajda,et al.  Assessing data quality: a computerized approach. , 1989, Social science & medicine.

[16]  J. Avorn,et al.  Increasing Levels of Restriction in Pharmacoepidemiologic Database Studies of Elderly and Comparison With Randomized Trial Results , 2007, Medical care.

[17]  Peter J. Neumann,et al.  Long-term persistence in use of statin therapy in elderly patients. , 2002, JAMA.

[18]  S. Suissa.  Immeasurable time bias in observational studies of drug effects on mortality. , 2008, American journal of epidemiology.

[19]  D A Grayson,et al.  Confounding confounding. , 1987, American journal of epidemiology.

[20]  R. Tamblyn,et al.  Validation of diagnostic codes within medical services claims. , 2004, Journal of clinical epidemiology.

[21]  R Neutra,et al.  Control of confounding in the assessment of medical technology. , 1980, International journal of epidemiology.

[22]  P. Montgomery,et al.  Pill Count, Self-Report, and Pharmacy Claims Data to Measure Medication Adherence in the Elderly , 1998, The Annals of pharmacotherapy.

[23]  K. Rothman.  Epidemiology: An Introduction , 2002 .

[24]  J Urquhart,et al.  Channeling bias in the interpretation of drug effects. , 1991, Statistics in medicine.

[25]  L. Abenhaim,et al.  A study of the effects of exposure misclassification due to the time-window design in pharmacoepidemiologic studies. , 1994, Journal of clinical epidemiology.

[26]  D. Kirking.  Comparison of medical records and prescription claims files in documenting prescription medication therapy , 1996 .

[27]  D. Redelmeier,et al.  The treatment of unrelated disorders in patients with chronic medical diseases. , 1998, The New England journal of medicine.

[28]  C R Weinberg,et al.  Toward a clearer definition of confounding. , 1993, American journal of epidemiology.

[29]  J. Avorn,et al.  Aging, comorbidity, and reduced rates of drug treatment for diabetes mellitus. , 1999, Journal of clinical epidemiology.

[30]  D. Mager,et al.  Relationship between generic and preferred-brand prescription copayment differentials and generic fill rate. , 2007, The American journal of managed care.

[31]  Richard Baumgartner,et al.  Using multiple data features improved the validity of osteoporosis case ascertainment from administrative databases. , 2008, Journal of clinical epidemiology.

[32]  Stephen MacMahon,et al.  Reliable assessment of the effects of treatment on mortality and major morbidity, II: observational studies , 2001, The Lancet.

[33]  S. Schneeweiss,et al.  Causation of Bias: The Episcope , 2001, Epidemiology.

[34]  R. Stern,et al.  Using a claims database to investigate drug-induced Stevens-Johnson syndrome. , 1991, Statistics in medicine.

[35]  A. Robinson.  I. Introduction , 1991 .

[36]  O. Miettinen,et al.  Confounding and effect-modification. , 1974, American journal of epidemiology.

[37]  S. Leatherman,et al.  Using Claims Data for Epidemiologic Research: The Concordance of Claims-Based Criteria With the Medical Record and Patient Survey for Identifying a Hypertensive Population , 1993, Medical care.

[38]  G. Howard.  On Validity , 1981 .

[39]  K. Rothman,et al.  Insights into different results from different causal contrasts in the presence of effect‐measure modification , 2006, Pharmacoepidemiology and drug safety.

[40]  Richard Barnett,et al.  Diabetes mellitus. , 1993, The Medical journal of Australia.

[41]  Michael L. Johnson,et al.  Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part III. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[42]  J. Avorn,et al.  A review of uses of health care utilization databases for epidemiologic research on therapeutics. , 2005, Journal of clinical epidemiology.

[43]  S. Shakir,et al.  An analysis of the exclusion criteria used in observational pharmacoepidemiological studies , 2007, Pharmacoepidemiology and drug safety.

[44]  L. Abenhaim,et al.  Utilization dynamics and risk comparisons in studies that use prescription information , 1994 .

[45]  D. Berry,et al.  Statistical models in epidemiology, the environment, and clinical trials , 2000 .

[46]  J. Avorn,et al.  Paradoxical Relations of Drug Treatment with Mortality in Older Persons , 2001, Epidemiology.

[47]  C. Lemière,et al.  A cohort study showed that health insurance databases were accurate to distinguish chronic obstructive pulmonary disease from asthma and classify disease severity. , 2005, Journal of clinical epidemiology.

[48]  R. Tamblyn,et al.  The use of prescription claims databases in pharmacoepidemiological research: the accuracy and comprehensiveness of the prescription claims database in Québec. , 1995, Journal of clinical epidemiology.

[49]  James M. Robins,et al.  Marginal Structural Models versus Structural nested Models as Tools for Causal inference , 2000 .

[50]  M. King,et al.  Matching prescription claims with medication data for nursing home residents: implications for prescriber feedback, drug utilisation studies and selection of prescription claims database. , 2001, Journal of clinical epidemiology.

[51]  I. Wilson,et al.  Antidepressant Use: Concordance Between Self-Report and Claims Records , 2003, Medical care.

[52]  G. Shaw,et al.  Maternal pesticide exposure from multiple sources and selected congenital anomalies. , 1999 .