Methods for safety signal detection in healthcare databases: a literature review

ABSTRACT Introduction: With increasing availability, the use of healthcare databases as complementary data source for drug safety signal detection has been explored to circumvent the limitations inherent in spontaneous reporting. Areas covered: To review the methods proposed for safety signal detection in healthcare databases and their performance. Expert opinion: Fifteen different data mining methods were identified. They are based on disproportionality analysis, traditional pharmacoepidemiological designs (e.g. self-controlled designs), sequence symmetry analysis (SSA), sequential statistical testing, temporal association rules, supervised machine learning (SML), and the tree-based scan statistic. When considering the performance of these methods, the self-controlled designs, the SSA, and the SML seemed the most interesting approaches. In the perspective of routine signal detection from healthcare databases, pragmatic aspects such as the need for stakeholders to understand the method in order to be confident in the results must be considered. From this point of view, the SSA could appear as the most suitable method for signal detection in healthcare databases owing to its simple principle and its ability to provide a risk estimate. However, further developments, such as automated prioritization, are needed to help stakeholders handle the multiplicity of signals.

[1]  J. Overhage,et al.  Advancing the Science for Active Surveillance: Rationale and Design for the Observational Medical Outcomes Partnership , 2010, Annals of Internal Medicine.

[2]  David Madigan,et al.  Empirical Performance of the Self-Controlled Case Series Design: Lessons for Developing a Risk Identification and Analysis System , 2013, Drug Safety.

[3]  H. Petri,et al.  Prescription sequence analysis: a new and fast method for assessing certain adverse reactions of prescription drugs in large populations. , 1988, Statistics in medicine.

[4]  M. Kulldorff,et al.  Early detection of adverse drug events within population‐based health networks: application of sequential testing methods , 2007, Pharmacoepidemiology and drug safety.

[5]  M. Kulldorff,et al.  A Maximized Sequential Probability Ratio Test for Drug and Vaccine Safety Surveillance , 2011 .

[6]  Uwe Aickelin,et al.  Signalling Paediatric Side Effects using an Ensemble of Simple Study Designs , 2014, Drug Safety.

[7]  Martijn J. Schuemie,et al.  Replication of the OMOP Experiment in Europe: Evaluating Methods for Risk Identification in Electronic Health Record Databases , 2013, Drug Safety.

[8]  A. Fourrier-Réglat,et al.  The EU-ADR project: preliminary results and perspective. , 2009, Studies in health technology and informatics.

[9]  J. Lam,et al.  Risk of Community-Acquired Pneumonia with Outpatient Proton-Pump Inhibitor Therapy: A Systematic Review and Meta-Analysis , 2015, PloS one.

[10]  N. Pratt,et al.  The Asian Pharmacoepidemiology Network (AsPEN): promoting multi‐national collaboration for pharmacoepidemiologic research in Asia , 2013, Pharmacoepidemiology and drug safety.

[11]  D. Madigan,et al.  A Comparison of the Empirical Performance of Methods for a Risk Identification System , 2013, Drug Safety.

[12]  Martijn J Schuemie,et al.  Methods for drug safety signal detection in longitudinal observational databases: LGPS and LEOPARD , 2011, Pharmacoepidemiology and drug safety.

[13]  Kristina Star,et al.  Outlier removal to uncover patterns in adverse drug reaction surveillance – a simple unmasking strategy , 2013, Pharmacoepidemiology and drug safety.

[14]  E. Roughead,et al.  Multi‐country rapid adverse drug event assessment: the Asian Pharmacoepidemiology Network (AsPEN) antipsychotic and acute hyperglycaemia study , 2013, Pharmacoepidemiology and drug safety.

[15]  David Madigan,et al.  Empirical Performance of the Case–Control Method: Lessons for Developing a Risk Identification and Analysis System , 2013, Drug Safety.

[16]  Yanqing Ji,et al.  A Potential Causal Association Mining Algorithm for Screening Adverse Drug Reactions in Postmarketing Surveillance , 2011, IEEE Transactions on Information Technology in Biomedicine.

[17]  E. Roughead,et al.  Sequence Symmetry Analysis as a Signal Detection Tool for Potential Heart Failure Adverse Events in an Administrative Claims Database , 2016, Drug Safety.

[18]  Martijn J Schuemie,et al.  Evaluating Performance of Risk Identification Methods Through a Large-Scale Simulation of Observational Data , 2013, Drug Safety.

[19]  W. DuMouchel,et al.  Novel Statistical Tools for Monitoring the Safety of Marketed Drugs , 2007, Clinical pharmacology and therapeutics.

[20]  A. Bate,et al.  A Bayesian neural network method for adverse drug reaction signal generation , 1998, European Journal of Clinical Pharmacology.

[21]  Jie Chen,et al.  Signaling Potential Adverse Drug Reactions from Administrative Health Databases , 2010, IEEE Transactions on Knowledge and Data Engineering.

[22]  N. Hartnell,et al.  Replication of the Weber Effect Using Postmarketing Adverse Event Reports Voluntarily Submitted to the United States Food and Drug Administration , 2004, Pharmacotherapy.

[23]  Uwe Aickelin,et al.  Comparison of algorithms that detect drug side effects using electronic healthcare databases , 2013, Soft Comput..

[24]  Uwe Aickelin,et al.  A Supervised Adverse Drug Reaction Signalling Framework Imitating Bradford Hill's Causality Considerations , 2015, J. Biomed. Informatics.

[25]  Signal detection of methylphenidate by comparing a spontaneous reporting database with a claims database. , 2011, Regulatory toxicology and pharmacology : RTP.

[26]  A. Pariente,et al.  Impact of Safety Alerts on Measures of Disproportionality in Spontaneous Reporting Databases The Notoriety Bias , 2007, Drug safety.

[27]  Richard Platt,et al.  The U.S. Food and Drug Administration's Mini‐Sentinel program: status and direction , 2012, Pharmacoepidemiology and drug safety.

[28]  Jesper Hallas,et al.  Adverse events with use of antiepileptic drugs: a prescription and event symmetry analysis , 2009, Pharmacoepidemiology and drug safety.

[29]  William DuMouchel,et al.  Bayesian Data Mining in Large Frequency Tables, with an Application to the FDA Spontaneous Reporting System , 1999 .

[30]  Andrea J Cook,et al.  Statistical approaches to group sequential monitoring of postmarket safety surveillance data: current state of the art for use in the Mini‐Sentinel pilot , 2012, Pharmacoepidemiology and drug safety.

[31]  S. Evans,et al.  Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports , 2001, Pharmacoepidemiology and drug safety.

[32]  Manfred Hauben,et al.  A conceptual approach to the masking effect of measures of disproportionality , 2014, Pharmacoepidemiology and drug safety.

[33]  Jie Chen,et al.  Mining Unexpected Associations for Signalling Potential Adverse Drug Reactions from Administrative Health Databases , 2006, PAKDD.

[34]  D. Madigan,et al.  Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership , 2012, Statistics in medicine.

[35]  F. Haramburu,et al.  Under-reporting of adverse drug reactions in general practice. , 2003, British journal of clinical pharmacology.

[36]  G. Niklas Norén,et al.  Good Signal Detection Practices: Evidence from IMI PROTECT , 2016, Drug Safety.

[37]  Sengwee Toh,et al.  Design and analysis choices for safety surveillance evaluations need to be tuned to the specifics of the hypothesized drug–outcome association , 2016, Pharmacoepidemiology and drug safety.

[38]  Charles E. Leonard,et al.  Comment on: “Desideratum for Evidence-Based Epidemiology” , 2014, Drug Safety.

[39]  Robert L Davis,et al.  Drug safety data mining with a tree‐based scan statistic , 2013, Pharmacoepidemiology and drug safety.

[40]  Marius Fieschi,et al.  Design and validation of an automated method to detect known adverse drug reactions in MEDLINE: a contribution from the EU-ADR project , 2013, J. Am. Medical Informatics Assoc..

[41]  Yea-Huei Kao Yang,et al.  Prescription sequence symmetry analysis: assessing risk, temporality, and consistency for adverse drug reactions across datasets in five countries , 2015, Pharmacoepidemiology and drug safety.

[42]  J. Montastruc,et al.  The nature of the scientific evidence leading to drug withdrawals for pharmacovigilance reasons in France , 2006, Pharmacoepidemiology and drug safety.

[43]  Yanqing Ji,et al.  A Method for Mining Infrequent Causal Associations and Its Application in Finding Adverse Drug Reaction Signal Pairs , 2013, IEEE Transactions on Knowledge and Data Engineering.

[44]  Jerry Avorn,et al.  Evaluating drug effects in the post-Vioxx world: there must be a better way. , 2006, Circulation.

[45]  J. Avorn,et al.  A review of uses of health care utilization databases for epidemiologic research on therapeutics. , 2005, Journal of clinical epidemiology.

[46]  Patrick Musonda,et al.  Tutorial in biostatistics: the self‐controlled case series method , 2006, Statistics in medicine.

[47]  Ju-Young Kim,et al.  Comparison and validation of data‐mining indices for signal detection: using the Korean national health insurance claims database , 2011, Pharmacoepidemiology and drug safety.

[48]  Jeffrey R Curtis,et al.  Adaptation of Bayesian Data Mining Algorithms to Longitudinal Claims Data: Coxib Safety as an Example , 2008, Medical care.

[49]  E. Roughead,et al.  The validity of sequence symmetry analysis (SSA) for adverse drug reaction signal detection , 2013, Pharmacoepidemiology and drug safety.

[50]  G. Niklas Norén,et al.  Temporal pattern discovery in longitudinal electronic patient records , 2010, Data Mining and Knowledge Discovery.

[51]  Ismaïl Ahmed,et al.  Class-imbalanced subsampling lasso algorithm for discovering adverse drug reactions , 2018, Statistical methods in medical research.

[52]  Comment on ‘Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership’ , 2013, Statistics in medicine.

[53]  David Madigan,et al.  Disproportionality methods for pharmacovigilance in longitudinal observational databases , 2013, Statistical methods in medical research.

[54]  Martijn J. Schuemie,et al.  A Reference Standard for Evaluation of Methods for Drug Safety Signal Detection Using Electronic Healthcare Record Databases , 2012, Drug Safety.

[55]  B Begaud,et al.  False-positives in spontaneous reporting: should we worry about them? , 1994, British journal of clinical pharmacology.

[56]  Lingling Li,et al.  A conditional sequential sampling procedure for drug safety surveillance , 2009, Statistics in medicine.

[57]  Bernard Bégaud,et al.  Rates of spontaneous reporting of adverse drug reactions in France. , 2002, JAMA.

[58]  S. Preston‐Martin,et al.  Head Injury as a Risk Factor for Brain Tumors in Children: Results from a Multicenter Case‐Control Study , 1996, Epidemiology.

[59]  Ismaïl Ahmed,et al.  A Method for the Minimization of Competition Bias in Signal Detection from Spontaneous Reporting Databases , 2016, Drug Safety.

[60]  F. Haramburu,et al.  Reports of hypoglycaemia associated with the use of ACE inhibitors and other drugs: a case/non-case study in the French pharmacovigilance system database. , 2003, British journal of clinical pharmacology.

[61]  J. Montastruc,et al.  Is spontaneous reporting always the most important information supporting drug withdrawals for pharmacovigilance reasons in France? , 2012, Pharmacoepidemiology and drug safety.

[62]  Patrick B. Ryan,et al.  Empirical Performance of a Self-Controlled Cohort Method: Lessons for Developing a Risk Identification and Analysis System , 2013, Drug Safety.

[63]  M. Kulldorff,et al.  A Tree‐Based Scan Statistic for Database Disease Surveillance , 2003, Biometrics.

[64]  M. Schuemie,et al.  Defining a Reference Set to Support Methodological Research in Drug Safety , 2013, Drug Safety.

[65]  G. Niklas Norén,et al.  A Decade of Data Mining and Still Counting , 2010, Drug safety.

[66]  Drummond Rennie,et al.  Postmarketing surveillance--lack of vigilance, lack of trust. , 2004, JAMA.

[67]  John Yen,et al.  A fuzzy logic-based computational recognition-primed decision model , 2007, Inf. Sci..

[68]  Martijn J. Schuemie,et al.  Using Electronic Health Care Records for Drug Safety Signal Detection: A Comparative Evaluation of Statistical Methods , 2012, Medical care.

[69]  Eric J Topol,et al.  Failing the public health--rofecoxib, Merck, and the FDA. , 2004, The New England journal of medicine.

[70]  David Madigan,et al.  Empirical Performance of a New User Cohort Method: Lessons for Developing a Risk Identification and Analysis System , 2013, Drug Safety.

[71]  Nicholas Moore.  The past, present and perhaps future of pharmacovigilance: homage to Folke Sjoqvist , 2013, European Journal of Clinical Pharmacology.

[72]  Seokyung Hahn,et al.  Signal detection of rosuvastatin compared to other statins: data‐mining study using national health insurance claims database , 2010, Pharmacoepidemiology and drug safety.

[73]  M. Loeb,et al.  Meta‐analysis: proton pump inhibitor use and the risk of community‐acquired pneumonia , 2010, Alimentary pharmacology & therapeutics.

[74]  Rok Blagus,et al.  Class prediction for high-dimensional class-imbalanced data , 2010, BMC Bioinformatics.

[75]  Johan Hopstadius,et al.  Safety surveillance of longitudinal databases: methodological considerations , 2011, Pharmacoepidemiology and drug safety.

[76]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[77]  Brent I. Fox,et al.  How Well Do Various Health Outcome Definitions Identify Appropriate Cases in Observational Studies? , 2013, Drug Safety.

[78]  J. Hallas.  Evidence of Depression Provoked by Cardiovascular Medication: A Prescription Sequence Symmetry Analysis , 1996, Epidemiology.

[79]  E. Roughead,et al.  The performance of sequence symmetry analysis as a tool for post-market surveillance of newly marketed medicines: a simulation study , 2014, BMC Medical Research Methodology.