What type of clinical evidence is needed to assess medical devices?

The objective of this mini-review is to discuss the role of real-world studies as a source of clinical evidence when experimental studies, such as randomised controlled trials (RCTs), are not available. Waiting for RCT evidence when the technology is diffusing could be anti-economical, inefficient from the policy perspective and methodologically questionable. We explain how real-world studies could provide relevant evidence to decision makers. Matching techniques are discussed as a viable solution for bias reduction. We describe a case study concerning a cost-effectiveness analysis based on real-world data of a technology already in use: Mitraclip combined with medical therapy versus medical therapy alone in patients with moderate-to-severe mitral regurgitation. The CEA has encountered the scepticism of most reviewers, due not to the statistical methodology but to the fact that the study was observational and not experimental. Editors and reviewers converged in considering real-world economic evaluations premature in the absence of a RCT, even if in the meantime the technology had been implanted >30 000 times. We believe there is a need to acknowledge the importance of real-world studies, and engage the scientific community in the promotion and use of clinical evidence produced through observational studies. Real-world data are a valid complement and/or alternative to RCTs to support policy decisions on medical devices http://ow.ly/SHsZ300pfCB

[1]  O. Baser,et al.  Too much ado about propensity score models? Comparing methods of propensity score matching. , 2006, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[2]  Mike Drummond,et al.  Generating appropriate clinical data for value assessment of medical devices: what role does regulation play? , 2014, Expert review of pharmacoeconomics & outcomes research.

[3]  Deborah Peikes,et al.  Propensity Score Matching: A Note of Caution for Evaluators of Social Programs , 2008 .

[4]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[5]  Peter C Austin,et al.  A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003 , 2008, Statistics in medicine.

[6]  Barbara Sianesi,et al.  What is propensity score matching , 2001 .

[7]  A. Boulton Registration and regulation of medical devices used in diabetes in Europe: need for radical reform. , 2013, The lancet. Diabetes & endocrinology.

[8]  R. Taylor,et al.  A more evidence based approach to the use of surrogate end points in policy making , 2011, BMJ : British Medical Journal.

[9]  Michael Drummond,et al.  Economic evaluation for devices and drugs--same or different? , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[10]  Pierfrancesco Agostoni,et al.  Are propensity scores really superior to standard multivariable analysis? , 2011, Contemporary clinical trials.

[11]  A. Maggioni,et al.  Real-world cost effectiveness of MitraClip combined with Medical Therapy Versus Medical therapy alone in patients with moderate or severe mitral regurgitation. , 2016, International journal of cardiology.

[12]  R. Torguson,et al.  Overview of the 2013 Food and Drug Administration Circulatory System Devices Panel Meeting on the MitraClip Delivery System , 2013, Circulation.

[13]  Michael L. Johnson,et al.  Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part III. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[14]  Peter C Austin,et al.  Some Methods of Propensity‐Score Matching had Superior Performance to Others: Results of an Empirical Investigation and Monte Carlo simulations , 2009, Biometrical journal. Biometrische Zeitschrift.

[15]  Aaron S Kesselheim,et al.  Regulation of medical devices in the United States and European Union. , 2012, The New England journal of medicine.

[16]  M. Drummond,et al.  Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions , 2013, ClinicoEconomics and outcomes research : CEOR.

[17]  L. Garrison,et al.  Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. , 2007, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[18]  Nancy A Dreyer,et al.  Why observational studies should be among the tools used in comparative effectiveness research. , 2010, Health affairs.

[19]  J. Robins,et al.  Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. , 2006, American journal of epidemiology.

[20]  Richard E Gliklich,et al.  GRACE principles: recognizing high-quality observational studies of comparative effectiveness. , 2010, The American journal of managed care.

[21]  Peter C Austin,et al.  Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement. , 2007, The Journal of thoracic and cardiovascular surgery.

[22]  Jennifer Hill,et al.  Discussion of research using propensity‐score matching: Comments on ‘A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003’ by Peter Austin, Statistics in Medicine , 2008, Statistics in medicine.

[23]  A. Das,et al.  Lack of publicly available scientific evidence on the safety and effectiveness of implanted medical devices. , 2014, JAMA internal medicine.

[24]  E. Foster,et al.  Acute and 12-month results with catheter-based mitral valve leaflet repair: the EVEREST II (Endovascular Valve Edge-to-Edge Repair) High Risk Study. , 2012, Journal of the American College of Cardiology.

[25]  Rajeev Dehejia,et al.  Propensity Score-Matching Methods for Nonexperimental Causal Studies , 2002, Review of Economics and Statistics.

[26]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[27]  T. Feldman,et al.  EVEREST II high risk study based UK cost-effectiveness analysis of MitraClip® in patients with severe mitral regurgitation ineligible for conventional repair/replacement surgery , 2013, Journal of medical economics.

[28]  David Atkins,et al.  Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part I. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[29]  T. Trikalinos,et al.  Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. , 2012, European heart journal.