Lessons from meta‐analyses of randomized clinical trials for analysis of distributed networks of observational databases

Networks of constellations of longitudinal observational databases, often electronic medical records or transactional insurance claims or both, are increasingly being used for studying the effects of medicinal products in real-world use. Such databases are frequently configured as distributed networks. That is, patient-level data are kept behind firewalls and not communicated outside of the data vendor other than in aggregate form. Instead, data are standardized across the network, and queries of the network are executed locally by data partners, and summary results provided to a central research partner(s) for amalgamation, aggregation, and summarization. Such networks can be huge covering years of data on upwards of 100 million patients. Examples of such networks include the FDA Sentinel Network, ASPEN, CNODES, and EU-ADR. As this is a new emerging field, we note in this paper the conceptual similarities and differences between the analysis of distributed networks and the now well-established field of meta-analysis of randomized clinical trials (RCTs). We recommend, wherever appropriate, to apply learnings from meta-analysis to help guide the development of distributed network analyses of longitudinal observational databases.

[1]  David Madigan,et al.  Good practices for real‐world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR‐ISPE Special Task Force on real‐world evidence in health care decision making , 2017, Pharmacoepidemiology and drug safety.

[2]  M. Epstein.  Guidelines for good pharmacoepidemiology practices (GPP) , 2005, Pharmacoepidemiology and drug safety.

[3]  M. Rawlins,et al.  Effects of intradermal bradykinin after inhibition of angiotensin converting enzyme. , 1987, British medical journal.

[4]  Julian P T Higgins,et al.  Recent developments in meta‐analysis , 2008, Statistics in medicine.

[5]  Douglas MacFadden,et al.  Application of Information Technology The Shared Health Research Information Network ( SHRINE ) : A Prototype Federated Query Tool for Clinical Data Repositories , 2014 .

[6]  G. Daniel,et al.  Commercial Insurance Databases , 2012 .

[7]  J. Hasford,et al.  Bridging Differences in Outcomes of Pharmacoepidemiological Studies: Design and First Results of the PROTECT Project , 2014, Current clinical pharmacology.

[8]  R Platt,et al.  Multicenter epidemiologic and health services research on therapeutics in the HMO Research Network Center for Education and Research on therapeutics , 2001, Pharmacoepidemiology and drug safety.

[9]  E. Roughead,et al.  Multi‐country rapid adverse drug event assessment: the Asian Pharmacoepidemiology Network (AsPEN) antipsychotic and acute hyperglycaemia study , 2013, Pharmacoepidemiology and drug safety.

[10]  David Madigan,et al.  Pooled analysis of rofecoxib placebo-controlled clinical trial data: lessons for postmarket pharmaceutical safety surveillance. , 2009, Archives of internal medicine.

[11]  Joseph M. Tonning,et al.  Perspectives on the Use of Data Mining in Pharmacovigilance , 2005, Drug safety.

[12]  Andrew Bate,et al.  A Comparative Assessment of Observational Medical Outcomes Partnership and Mini-Sentinel Common Data Models and Analytics: Implications for Active Drug Safety Surveillance , 2015, Drug Safety.

[13]  Bruce M Psaty,et al.  Detection, verification, and quantification of adverse drug reactions , 2004, BMJ : British Medical Journal.

[14]  Steven J. Jacobsen,et al.  The Vaccine Safety Datalink: A Model for Monitoring Immunization Safety , 2011, Pediatrics.

[15]  S. Ikeda,et al.  Development of a Database of Health Insurance Claims: Standardization of Disease Classifications and Anonymous Record Linkage , 2010, Journal of epidemiology.

[16]  A. Bate,et al.  From Big Data to Smart Data for Pharmacovigilance: The Role of Healthcare Databases and Other Emerging Sources , 2018, Drug Safety.

[17]  Sebastian Schneeweiss,et al.  Using high‐dimensional propensity scores to automate confounding control in a distributed medical product safety surveillance system , 2012, Pharmacoepidemiology and drug safety.

[18]  Marjanka K. Schmidt,et al.  Data management and data analysis techniques in pharmacoepidemiological studies using a pre‐planned multi‐database approach: a systematic literature review† , 2015, Pharmacoepidemiology and drug safety.

[19]  Andrew Bate,et al.  An Evaluation of the THIN Database in the OMOP Common Data Model for Active Drug Safety Surveillance , 2013, Drug Safety.

[20]  K. Bhaskaran,et al.  Data Resource Profile: Clinical Practice Research Datalink (CPRD) , 2015, International journal of epidemiology.

[21]  G A Colditz,et al.  The role of meta-analysis in the regulatory process for foods, drugs, and devices. , 1999, JAMA.

[22]  D. Madigan,et al.  Evaluating the impact of database heterogeneity on observational study results. , 2013, American journal of epidemiology.

[23]  I. Olkin,et al.  Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement , 1999, The Lancet.

[24]  J. Avorn,et al.  A review of uses of health care utilization databases for epidemiologic research on therapeutics. , 2005, Journal of clinical epidemiology.

[25]  K. Blake,et al.  Increasing scientific standards, independence and transparency in post‐authorisation studies: the role of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance , 2012, Pharmacoepidemiology and drug safety.

[26]  Patrick B. Ryan,et al.  Development and evaluation of a common data model enabling active drug safety surveillance using disparate healthcare databases , 2010, J. Am. Medical Informatics Assoc..

[27]  R. Platt,et al.  Distributed Health Data Networks: A Practical and Preferred Approach to Multi-Institutional Evaluations of Comparative Effectiveness, Safety, and Quality of Care , 2010, Medical care.

[28]  R. Platt,et al.  Developing the Sentinel System--a national resource for evidence development. , 2011, The New England journal of medicine.

[29]  A. Bate.  Guidance to reinforce the credibility of health care database studies and ensure their appropriate impact , 2017, Pharmacoepidemiology and drug safety.

[30]  A. Bate,et al.  Incidence Rates and Trends of Hip/Femur Fractures in Five European Countries: Comparison Using E-Healthcare Records Databases , 2014, Calcified Tissue International.

[31]  Richard Platt,et al.  The U.S. Food and Drug Administration's Mini‐Sentinel program: status and direction , 2012, Pharmacoepidemiology and drug safety.

[32]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[33]  M. Schuemie,et al.  Combining electronic healthcare databases in Europe to allow for large‐scale drug safety monitoring: the EU‐ADR Project , 2011, Pharmacoepidemiology and drug safety.

[34]  Richard Platt,et al.  Launching PCORnet, a national patient-centered clinical research network , 2014, Journal of the American Medical Informatics Association : JAMIA.

[35]  S. Wood,et al.  Angio-oedema and urticaria associated with angiotensin converting enzyme inhibitors. , 1987, British medical journal.

[36]  Marsha E Reichman,et al.  Dabigatran and postmarketing reports of bleeding. , 2013, The New England journal of medicine.

[37]  Olaf Klungel,et al.  Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0 , 2017, Pharmacoepidemiology and drug safety.

[38]  Rongjun Shen,et al.  Computer-assisted expert case definition in electronic health records , 2016, Int. J. Medical Informatics.

[39]  Brian Sauer,et al.  Guidelines for good database selection and use in pharmacoepidemiology research , 2012, Pharmacoepidemiology and drug safety.

[40]  A. Bate,et al.  Evidence generation from healthcare databases: recommendations for managing change , 2016, Pharmacoepidemiology and drug safety.

[41]  J. Overhage,et al.  Advancing the Science for Active Surveillance: Rationale and Design for the Observational Medical Outcomes Partnership , 2010, Annals of Internal Medicine.

[42]  J. Ioannidis,et al.  Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials , 2011, BMJ : British Medical Journal.

[43]  Xavier Kurz,et al.  Multi‐centre, multi‐database studies with common protocols: lessons learnt from the IMI PROTECT project , 2016, Pharmacoepidemiology and drug safety.

[44]  I. Olkin,et al.  Meta-analysis of observational studies in epidemiology - A proposal for reporting , 2000 .

[45]  D. Madigan,et al.  Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership , 2012, Statistics in medicine.

[46]  Richard Platt,et al.  Four health data networks illustrate the potential for a shared national multipurpose big-data network. , 2014, Health affairs.

[47]  Andrew Bate,et al.  Designing and incorporating a real world data approach to international drug development and use: what the UK offers. , 2016, Drug discovery today.

[48]  Adrian F Hernandez,et al.  Comparative risk for angioedema associated with the use of drugs that target the renin-angiotensin-aldosterone system. , 2012, Archives of internal medicine.

[49]  J. Lei,et al.  Combining multiple healthcare databases for postmarketing drug and vaccine safety surveillance: why and how? , 2014, Journal of internal medicine.

[50]  Jonathan J Deeks,et al.  Much ado about nothing: a comparison of the performance of meta‐analytical methods with rare events , 2007, Statistics in medicine.

[51]  A. Walker.  Orthogonal predictions: follow‐up questions for suggestive data , 2010, Pharmacoepidemiology and drug safety.

[52]  David Atkins,et al.  Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part I. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[53]  Norman E. Breslow,et al.  Odds ratio estimators when the data are sparse , 1981 .

[54]  Richard Platt,et al.  The organizational structure and governing principles of the Food and Drug Administration's Mini‐Sentinel pilot program , 2012, Pharmacoepidemiology and drug safety.

[55]  E. Lewis,et al.  Measles-mumps-rubella and other measles-containing vaccines do not increase the risk for inflammatory bowel disease: a case-control study from the Vaccine Safety Datalink project. , 2001, Archives of pediatrics & adolescent medicine.

[56]  Marsha A Raebel,et al.  Design considerations, architecture, and use of the Mini‐Sentinel distributed data system , 2012, Pharmacoepidemiology and drug safety.

[57]  R. Platt,et al.  The new Sentinel Network--improving the evidence of medical-product safety. , 2009, The New England journal of medicine.

[58]  Andrew Bate,et al.  Do FDA label changes work? Assessment of the 2010 class label change for proton pump inhibitors using the Sentinel System's analytic tools , 2018, Pharmacoepidemiology and drug safety.

[59]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[60]  N. Pratt,et al.  The Asian Pharmacoepidemiology Network (AsPEN): promoting multi‐national collaboration for pharmacoepidemiologic research in Asia , 2013, Pharmacoepidemiology and drug safety.

[61]  A. Gould.  Generating and confirming hypotheses , 2010, Pharmacoepidemiology and drug safety.

[62]  P. Ravani,et al.  Proton pump inhibitors and the risk of hospitalisation for community-acquired pneumonia: replicated cohort studies with meta-analysis , 2013, Gut.