Developing an expert panel process to refine health outcome definitions in observational data

OBJECTIVES Drug safety surveillance using observational data requires valid adverse event, or health outcome of interest (HOI) measurement. The objectives of this study were to develop a method to review HOI definitions in claims databases using (1) web-based digital tools to present de-identified patient data, (2) a systematic expert panel review process, and (3) a data collection process enabling analysis of concepts-of-interest that influence panelists' determination of HOI. METHODS De-identified patient data were presented via an interactive web-based dashboard to enable case review and determine if specific HOIs were present or absent. Criteria for determining HOIs and their severity were provided to each panelist. Using a modified Delphi method, six panelist pairs independently reviewed approximately 200 cases across each of three HOIs (acute liver injury, acute kidney injury, and acute myocardial infarction) such that panelist pairs independently reviewed the same cases. Panelists completed an assessment within the dashboard for each case that included their assessment of the presence or absence of the HOI, HOI severity (if present), and data contributing to their decision. Discrepancies within panelist pairs were resolved during a consensus process. RESULTS Dashboard development was iterative, focusing on data presentation and recording panelists' assessments. Panelists reported quickly learning how to use the dashboard. The assessment module was used consistently. The dashboard was reliable, enabling an efficient review process for panelists. Modifications were made to the dashboard and review process when necessary to facilitate case review. Our methods should be applied to other health outcomes of interest to further refine the dashboard and case review process. CONCLUSION The expert review process was effective and was supported by the web-based dashboard. Our methods for case review and classification can be applied to future methods for case identification in observational data sources.

[1]  Sandra L Kane-Gill,et al.  Adverse Drug Reactions in Hospital and Ambulatory Care Settings Identified Using a Large Administrative Database , 2010, The Annals of pharmacotherapy.

[2]  Patrick B. Ryan,et al.  Health Outcomes of Interest in Observational Data: Issues in Identifying Definitions in the Literature , 2012 .

[3]  B. Strom,et al.  Data validity issues in using claims data , 2001, Pharmacoepidemiology and drug safety.

[4]  Denys T. Lau,et al.  Evaluation of serious adverse drug reactions: a proactive pharmacovigilance program (RADAR) vs safety activities conducted by the Food and Drug Administration and pharmaceutical manufacturers. , 2007, Archives of internal medicine.

[5]  J Hasford,et al.  Adverse drug reaction monitoring--cost and benefit considerations. Part I: frequency of adverse drug reactions causing hospital admissions. , 1997, Pharmacoepidemiology and drug safety.

[6]  D Bajpai,et al.  Evaluation of the prevalence and economic burden of adverse drug reactions presenting to the medical emergency department of a tertiary referral centre: a prospective study , 2007, BMC clinical pharmacology.

[7]  Patrick B. Ryan,et al.  Validation of a common data model for active safety surveillance research , 2012, J. Am. Medical Informatics Assoc..

[8]  Daniel J. Vreeman,et al.  Logical Observation Identifiers Names and Codes (LOINC®) users' guide , 2010 .

[9]  N. Moore,et al.  Frequency and cost of serious adverse drug reactions in a department of general medicine. , 1998, British journal of clinical pharmacology.

[10]  Josip Car,et al.  Trends in hospital admissions for adverse drug reactions in England: analysis of national hospital episode statistics 1998–2005 , 2007, BMC clinical pharmacology.

[11]  Frederick W. Fraunfelder,et al.  Adverse ocular drug reactions recently identified by the National Registry of Drug-Induced Ocular Side Effects. , 2004, Ophthalmology.

[12]  J. Overhage,et al.  Advancing the Science for Active Surveillance: Rationale and Design for the Observational Medical Outcomes Partnership , 2010, Annals of Internal Medicine.

[13]  Robert L Davis,et al.  Real-Time Vaccine Safety Surveillance for the Early Detection of Adverse Events , 2007, Medical care.

[14]  R. Platt,et al.  Developing the Sentinel System--a national resource for evidence development. , 2011, The New England journal of medicine.

[15]  William M. Lee,et al.  Acetaminophen‐induced acute liver failure: Results of a United States multicenter, prospective study , 2005, Hepatology.

[16]  J Hasford,et al.  Adverse drug reaction monitoring—cost and benefit considerations. Part II: cost and preventability of adverse drug reactions leading to hospital admission , 1997, Pharmacoepidemiology and drug safety.

[17]  M. Sordo,et al.  Rapid Identification of Myocardial Infarction Risk Associated With Diabetes Medications Using Electronic Medical Records , 2009, Diabetes Care.

[18]  A. Grizzle,et al.  Drug-related morbidity and mortality: updating the cost-of-illness model. , 2001, Journal of the American Pharmaceutical Association.