An overview of methodological flaws of real-world studies investigating drug safety in the post-marketing setting

ABSTRACT Introduction The evaluation of the post-marketing safety profile of drugs is a continuous monitoring process for approved and marketed medicines and it is crucial for detecting new adverse drug reactions. As such, real-world studies are essential to complement pre-marketing evidence with information concerning drug risk-benefit profile and use in wider patient populations and they have a great potential to support post-marketing drug safety evaluations. Areas covered A detailed description of the main limitations of real-world data sources (i.e. claims databases, electronic healthcare records, drug/disease registers and spontaneous reporting system databases) and of the main methodological challenges of real-world studies in generating real-world evidence is provided. Expert opinion Real-world evidence biases can be ascribed to both the methodological approach and the specific limitations of the different real-world data sources used to carry out the study. As such, it is crucial to characterize the quality of real-world data, by establishing guidelines and best practices for the assessment of data fitness for purpose. On the other hand, it is important that real-world studies are conducted using a rigorous methodology, aimed at minimizing the risk of bias.

[1]  G. Trifirò,et al.  Is PEGylation of Drugs Associated with Hypersensitivity Reactions? An Analysis of the Italian National Spontaneous Adverse Drug Reaction Reporting System , 2023, Drug Safety.

[2]  A. Bitton,et al.  Proton pump inhibitors and the risk of inflammatory bowel disease: population-based cohort study , 2023, Gut.

[3]  Z. Babar,et al.  Pharmacovigilance in High-Income Countries: Current Developments and a Review of Literature , 2023, Pharmacy.

[4]  M. Hauben,et al.  More extreme duplication in FDA Adverse Event Reporting System detected by literature reference normalization and fuzzy string matching , 2022, Pharmacoepidemiology and drug safety.

[5]  Demosthenes Panagiotakos,et al.  Real-world data: a brief review of the methods, applications, challenges and opportunities , 2022, BMC Medical Research Methodology.

[6]  Wai Hoong Chang,et al.  Comparing clinical trial population representativeness to real-world populations: an external validity analysis encompassing 43 895 trials and 5 685 738 individuals across 989 unique drugs and 286 conditions in England. , 2022, The Lancet. Healthy longevity.

[7]  Robert W. Platt,et al.  Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions , 2022, Nature Communications.

[8]  J. Concato,et al.  Real-World Evidence - Where Are We Now? , 2022, The New England journal of medicine.

[9]  G. Trifirò,et al.  A New Era of Pharmacovigilance: Future Challenges and Opportunities , 2022, Frontiers in Drug Safety and Regulation.

[10]  F. Rockhold,et al.  Real-World Evidence for Regulatory Decision-Making: Guidance From Around the World. , 2022, Clinical therapeutics.

[11]  W. Bolislis,et al.  Real-World Data in the Postapproval Setting as Applied by the EMA and the US FDA. , 2022, Clinical therapeutics.

[12]  J. Lund,et al.  An overview of real‐world data sources for oncology and considerations for research , 2021, CA: a cancer journal for clinicians.

[13]  G. Trifirò,et al.  COVID-19 Patient Management in Outpatient Setting: A Population-Based Study from Southern Italy , 2021, Journal of clinical medicine.

[14]  J. Rassen,et al.  The Role of Real‐World Evidence in FDA‐Approved New Drug and Biologics License Applications , 2021, Clinical pharmacology and therapeutics.

[15]  C. Quinten,et al.  Marketing Authorization Applications Made to the European Medicines Agency in 2018–2019: What was the Contribution of Real‐World Evidence? , 2021, Clinical pharmacology and therapeutics.

[16]  A. Bérard Pharmacoepidemiology Research-Real-World Evidence for Decision Making , 2021, Frontiers in Pharmacology.

[17]  K. Verhamme,et al.  Drug-induced Urinary Retention: An Analysis of a National Spontaneous Adverse Drug Reaction Reporting Database. , 2021, European urology focus.

[18]  J. Franklin,et al.  Prevalence of Avoidable and Bias‐Inflicting Methodological Pitfalls in Real‐World Studies of Medication Safety and Effectiveness , 2021, Clinical pharmacology and therapeutics.

[19]  K. Hiramatsu,et al.  Current Status, Challenges, and Future Perspectives of Real-World Data and Real-World Evidence in Japan , 2021, Drugs - Real World Outcomes.

[20]  E. Perucca,et al.  Pediatric adverse reactions to antiseizure medications – An analysis of data from the Italian spontaneous reporting system (2001–2019) , 2021, Epilepsy & Behavior.

[21]  Stefano E. Rensi,et al.  A New Era in Pharmacovigilance: Toward Real‐World Data and Digital Monitoring , 2021, Clinical pharmacology and therapeutics.

[22]  Jens Torup Østby,et al.  Common Problems, Common Data Model Solutions: Evidence Generation for Health Technology Assessment , 2020, PharmacoEconomics.

[23]  G. Trifirò,et al.  Real World Use of Antidiabetic Drugs in the Years 2011–2017: A Population-Based Study from Southern Italy , 2020, International journal of environmental research and public health.

[24]  S. Shakir,et al.  Postmarketing studies: can they provide a safety net for COVID-19 vaccines in the UK? , 2020, BMJ Evidence-Based Medicine.

[25]  Rosa Gini,et al.  Different Strategies to Execute Multi‐Database Studies for Medicines Surveillance in Real‐World Setting: A Reflection on the European Model , 2020, Clinical pharmacology and therapeutics.

[26]  E. Russo,et al.  Safety Profile of Biologics Used in Rheumatology: An Italian Prospective Pharmacovigilance Study , 2020, Journal of clinical medicine.

[27]  T. Stürmer,et al.  Methodological considerations when analysing and interpreting real-world data. , 2020, Rheumatology.

[28]  Jessica M Franklin,et al.  Alternative approaches for confounding adjustment in observational studies using weighting based on the propensity score: a primer for practitioners , 2019, BMJ.

[29]  G. Trifirò,et al.  Measuring the Effectiveness of Safety Warnings on the Risk of Stroke in Older Antipsychotic Users: A Nationwide Cohort Study in Two Large Electronic Medical Records Databases in the United Kingdom and Italy , 2019, Drug Safety.

[30]  Y. Ho,et al.  The Advantages and Challenges of Using Real‐World Data for Patient Care , 2019, Clinical and translational science.

[31]  L. Mantovani,et al.  Role of healthcare databases and registries for surveillance of orphan drugs in the real-world setting: the Italian case study , 2019, Expert opinion on drug safety.

[32]  J. H. van der Lee,et al.  The patient’s view on rare disease trial design – a qualitative study , 2019, Orphanet Journal of Rare Diseases.

[33]  J. Faillie Case-non-case studies: Principle, methods, bias and interpretation. , 2019, Therapie.

[34]  G. Trifirò,et al.  All-cause mortality and antipsychotic use among elderly persons with high baseline cardiovascular and cerebrovascular risk: a multi-center retrospective cohort study in Italy , 2019, Expert opinion on drug metabolism & toxicology.

[35]  A. Dubey,et al.  Postmarket surveillance: a review on key aspects and measures on the effective functioning in the context of the United Kingdom and Canada , 2019, Therapeutic advances in drug safety.

[36]  L. Hemkens How Routinely Collected Data for Randomized Trials Provide Long-term Randomized Real-World Evidence. , 2018, JAMA network open.

[37]  G. Corrao,et al.  Adherence of Elderly Patients with Cardiovascular Disease to Statins and the Risk of Exacerbation of Chronic Obstructive Pulmonary Disease: Evidence from an Italian Real-World Investigation , 2018, Drugs & Aging.

[38]  R. Caporali,et al.  Randomized controlled trials and real-world data: differences and similarities to untangle literature data. , 2018, Rheumatology.

[39]  L. Naldi,et al.  The Role of European Healthcare Databases for Post-Marketing Drug Effectiveness, Safety and Value Evaluation: Where Does Italy Stand? , 2018, Drug Safety.

[40]  M. Davoli,et al.  Neonatal outcomes following new reimbursement limitations on palivizumab in Italy , 2018, Archives of Disease in Childhood.

[41]  Sania Amr,et al.  Postmarket Safety Outcomes for New Molecular Entity (NME) Drugs Approved by the Food and Drug Administration Between 2002 and 2014 , 2018, Clinical pharmacology and therapeutics.

[42]  J. K. Bower,et al.  Addressing Bias in Electronic Health Record-based Surveillance of Cardiovascular Disease Risk: Finding the Signal Through the Noise , 2017, Current Epidemiology Reports.

[43]  D. Vohora,et al.  Relevance of the Weber effect in contemporary pharmacovigilance of oncology drugs , 2017, Therapeutics and clinical risk management.

[44]  G. Trifirò,et al.  Overview of the Safety of Anti-VEGF Drugs: Analysis of the Italian Spontaneous Reporting System , 2017, Drug Safety.

[45]  F. Lapi,et al.  Hepatitis C virus infection: opportunities for an earlier detection in primary care , 2017, European journal of gastroenterology & hepatology.

[46]  P. Doshi,et al.  Contribution of industry funded post-marketing studies to drug safety: survey of notifications submitted to regulatory agencies , 2017, British Medical Journal.

[47]  B. Stricker,et al.  Antibiotic-Induced Liver Injury in Paediatric Outpatients: A Case-Control Study in Primary Care Databases , 2016, Drug Safety.

[48]  Xin-ji Zhang,et al.  Overview of phase IV clinical trials for postmarket drug safety surveillance: a status report from the ClinicalTrials.gov registry , 2016, BMJ Open.

[49]  M. Yotebieng,et al.  Targeted Spontaneous Reporting: Assessing Opportunities to Conduct Routine Pharmacovigilance for Antiretroviral Treatment on an International Scale , 2016, Drug Safety.

[50]  Alaa M Althubaiti,et al.  Information bias in health research: definition, pitfalls, and adjustment methods , 2016, Journal of multidisciplinary healthcare.

[51]  L. G. García Rodríguez,et al.  Validation of low‐dose aspirin prescription data in The Health Improvement Network: how much misclassification due to over‐the‐counter use? , 2015, Pharmacoepidemiology and drug safety.

[52]  G. Trifirò,et al.  Association of Individual Non-Steroidal Anti-Inflammatory Drugs and Chronic Kidney Disease: A Population-Based Case Control Study , 2015, PloS one.

[53]  E. Russo,et al.  Limitations and obstacles of the spontaneous adverse drugs reactions reporting: Two “challenging” case reports , 2013, Journal of pharmacology & pharmacotherapeutics.

[54]  D. Madigan,et al.  Evaluating the impact of database heterogeneity on observational study results. , 2013, American journal of epidemiology.

[55]  N. Magrini,et al.  [A perspective for the role of drug registries in the post-marketing surveillance]. , 2013, Recenti progressi in medicina.

[56]  D. Madigan,et al.  Does design matter? Systematic evaluation of the impact of analytical choices on effect estimates in observational studies , 2013, Therapeutic advances in drug safety.

[57]  Miriam Sturkenboom,et al.  Postmarketing Safety Surveillance , 2013, Drug Safety.

[58]  S. Pal,et al.  WHO Strategy for Collecting Safety Data in Public Health Programmes: Complementing Spontaneous Reporting Systems , 2013, Drug Safety.

[59]  Samy Suissa,et al.  Metformin and the Risk of Cancer , 2012, Diabetes Care.

[60]  M. Hernán,et al.  Systematic Reviews and Meta-and Pooled Analyses Bias in Observational Studies of Prevalent Users : Lessons for Comparative Effectiveness Research From a Meta-Analysis of Statins , 2012 .

[61]  Brian Sauer,et al.  Guidelines for good database selection and use in pharmacoepidemiology research , 2012, Pharmacoepidemiology and drug safety.

[62]  Samy Suissa,et al.  Immortal time bias in pharmaco-epidemiology. , 2008, American journal of epidemiology.

[63]  Samy Suissa,et al.  Immortal time bias in observational studies of drug effects , 2007, Pharmacoepidemiology and drug safety.

[64]  Moride Yola,et al.  Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. , 1994 .

[65]  W. Inman,et al.  Under-reporting of adverse drug reactions. , 1985, British medical journal.

[66]  A. Zwicky Heads , 1985, Journal of Linguistics.

[67]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[68]  A. Feinstein,et al.  The problem of "protopathic bias" in case-control studies. , 1980, The American journal of medicine.

[69]  G. Trifirò,et al.  [The role of real-world evidence for the study of rare diseases epidemiology and the post-marketing evaluation of orphan drugs.] , 2022, Recenti progressi in medicina.

[70]  P. Cutroneo,et al.  Safety and Tolerability of Antipsychotic Drugs in Pediatric Patients: Data From a 1-Year Naturalistic Study , 2020, Frontiers in Psychiatry.

[71]  Mario Prost System , 2019, Concepts for International Law.

[72]  C. L. Ventola Big Data and Pharmacovigilance: Data Mining for Adverse Drug Events and Interactions. , 2018, P & T : a peer-reviewed journal for formulary management.

[73]  Preciosa M. ColomaGianluca Trifiro Where does Signal Detection Using Electronic Healthcare Records Fit into the Big Picture , 2013 .

[74]  L. Hazell,et al.  Under-Reporting of Adverse Drug Reactions , 2006, Drug safety.

[75]  J. Samet,et al.  Food and Drug Administration , 2007, BMJ : British Medical Journal.

[76]  Y. Moride,et al.  Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. , 1994, Journal of clinical epidemiology.

[77]  S. Coughlin Recall bias in epidemiologic studies. , 1990, Journal of clinical epidemiology.