A Comparative Assessment of Observational Medical Outcomes Partnership and Mini-Sentinel Common Data Models and Analytics: Implications for Active Drug Safety Surveillance

AbstractIntroductionAn often key component to coordinating surveillance activities across distributed networks is the design and implementation of a common data model (CDM). The purpose of this study was to evaluate two drug safety surveillance CDMs from an ecosystem perspective to better understand how differences in CDMs and analytic tools affect usability and interpretation of results.MethodsHumana claims data from 2007 to 2012 were mapped to Observational Medical Outcomes Partnership (OMOP) and Mini-Sentinel CDMs. Data were described and compared at the patient level by source code and mapped concepts. Study cohort construction and effect estimates were also compared using two different analytical methods—one based on a new user design implementing a high-dimensional propensity score (HDPS) algorithm and the other based on univariate self-controlled case series (SCCS) design—across six established positive drug-outcome pairs to learn how differences in CDMs and analytics influence steps in the database analytic process and results.ResultsClaims data for approximately 7.7 million Humana health plan members were transformed into the two CDMs. Three health outcome cohorts and two drug cohorts showed differences in cohort size and constituency between Mini-Sentinel and OMOP CDMs, which was a result of multiple factors. Overall, the implementation of the HDPS procedure on Mini-Sentinel CDM detected more known positive associations than that on OMOP CDM. The SCCS method results were comparable on both CDMs. Differences in the implementation of the HDPS procedure between the two CDMs were identified; analytic model and risk period specification had a significant impact on the performance of the HDPS procedure on OMOP CDM.ConclusionsDifferences were observed between OMOP and Mini-Sentinel CDMs. The analysis of both CDMs at the data model level indicated that such conceptual differences had only a slight but not significant impact on identifying known safety associations. Our results show that differences at the ecosystem level of analyses across the CDMs can lead to strikingly different risk estimations, but this can be primarily attributed to the choices of analytic approach and their implementation in the community-developed analytic tools. The opportunities of using CDMs are clear, but our study shows the need for judicious comparison of analyses across the CDMs. Our work emphasizes the need for ongoing efforts to ensure sustainable transparent platforms to maintain and develop CDMs and associated tools for effective safety surveillance.

[1]  Barbara Evans,et al.  A policy framework for public health uses of electronic health data , 2012, Pharmacoepidemiology and drug safety.

[2]  R. Platt,et al.  Developing the Sentinel System--a national resource for evidence development. , 2011, The New England journal of medicine.

[3]  J. Avorn,et al.  High-dimensional Propensity Score Adjustment in Studies of Treatment Effects Using Health Care Claims Data , 2009, Epidemiology.

[4]  M. Suchard,et al.  The Impact of Drug and Outcome Prevalence on the Feasibility and Performance of Analytical Methods for a Risk Identification and Analysis System , 2013, Drug Safety.

[5]  Marsha A Raebel,et al.  Design considerations, architecture, and use of the Mini‐Sentinel distributed data system , 2012, Pharmacoepidemiology and drug safety.

[6]  D. Madigan,et al.  Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership , 2012, Statistics in medicine.

[7]  Jeremy A Rassen,et al.  Active Safety Monitoring of New Medical Products Using Electronic Healthcare Data: Selecting Alerting Rules , 2012, Epidemiology.

[8]  Ralph B D'Agostino,et al.  Re‐using Mini‐Sentinel data following rapid assessments of potential safety signals via modular analytic programs , 2013, Pharmacoepidemiology and drug safety.

[9]  M. Schuemie,et al.  Defining a Reference Set to Support Methodological Research in Drug Safety , 2013, Drug Safety.

[10]  Martijn J. Schuemie,et al.  A Reference Standard for Evaluation of Methods for Drug Safety Signal Detection Using Electronic Healthcare Record Databases , 2012, Drug Safety.

[11]  Sengwee Toh,et al.  Validity of diagnostic codes to identify cases of severe acute liver injury in the U.S. Food and Drug Administration's Mini‐Sentinel Distributed Database , 2013, Pharmacoepidemiology and drug safety.

[12]  R. Platt,et al.  Distributed Health Data Networks: A Practical and Preferred Approach to Multi-Institutional Evaluations of Comparative Effectiveness, Safety, and Quality of Care , 2010, Medical care.

[13]  Johan Hopstadius,et al.  Safety surveillance of longitudinal databases: methodological considerations , 2011, Pharmacoepidemiology and drug safety.

[14]  David Madigan,et al.  Empirical Performance of a New User Cohort Method: Lessons for Developing a Risk Identification and Analysis System , 2013, Drug Safety.

[15]  M. Reynolds,et al.  A systematic review of validated methods for identifying anaphylaxis, including anaphylactic shock and angioneurotic edema, using administrative and claims data , 2012, Pharmacoepidemiology and drug safety.

[16]  Andrew Bate,et al.  An Evaluation of the THIN Database in the OMOP Common Data Model for Active Drug Safety Surveillance , 2013, Drug Safety.

[17]  J. Overhage,et al.  Advancing the Science for Active Surveillance: Rationale and Design for the Observational Medical Outcomes Partnership , 2010, Annals of Internal Medicine.

[18]  Samy Suissa,et al.  Primer: administrative health databases in observational studies of drug effects—advantages and disadvantages , 2007, Nature Clinical Practice Rheumatology.

[19]  Heather J Whitaker,et al.  The methodology of self-controlled case series studies , 2009, Statistical methods in medical research.

[20]  Patrick B. Ryan,et al.  Validation of a common data model for active safety surveillance research , 2012, J. Am. Medical Informatics Assoc..