Why caution is recommended with post-hoc individual patient matching for estimation of treatment effect in parallel-group randomized controlled trials: the case of acute stroke trials.

A post-hoc individual patient matching procedure was recently proposed within the context of parallel group randomized clinical trials (RCTs) as a method for estimating treatment effect. In this paper, we consider a post-hoc individual patient matching problem within a parallel group RCT as a multi-objective decision-making problem focussing on the trade-off between the quality of individual matches and the overall percentage of matching. Using acute stroke trials as a context, we utilize exact optimization and simulation techniques to investigate a complex relationship between the overall percentage of individual post-hoc matching, the size of the respective RCT, and the quality of matching on variables highly prognostic for a good functional outcome after stroke, as well as the dispersion in these variables. It is empirically confirmed that a high percentage of individual post-hoc matching can only be achieved when the differences in prognostic baseline variables between individually matched subjects within the same pair are sufficiently large and that the unmatched subjects are qualitatively different to the matched ones. It is concluded that the post-hoc individual matching as a technique for treatment effect estimation in parallel-group RCTs should be exercised with caution because of its propensity to introduce significant bias and reduce validity. If used with appropriate caution and thorough evaluation, this approach can complement other viable alternative approaches for estimating the treatment effect.

[1]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[2]  G. Guyatt,et al.  Interpreting treatment effects in randomised trials , 1998, BMJ.

[3]  Jasjeet S. Sekhon,et al.  Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching Package for R , 2008 .

[4]  C. Weinberg,et al.  Paired versus two-sample design for a clinical trial of treatments with dichotomous outcome: power considerations. , 1982, Biometrics.

[5]  D. Rubin,et al.  The Bias Due to Incomplete Matching , 1985 .

[6]  Marco Caliendo,et al.  Some Practical Guidance for the Implementation of Propensity Score Matching , 2005, SSRN Electronic Journal.

[7]  Peter C Austin,et al.  A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003 , 2008, Statistics in medicine.

[8]  Gregory W Albers,et al.  Time to treatment with intravenous alteplase and outcome in stroke: an updated pooled analysis of ECASS, ATLANTIS, NINDS, and EPITHET trials , 2010, The Lancet.

[9]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[10]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[11]  E. Dettmann,et al.  Distance functions for matching in small samples , 2011, Comput. Stat. Data Anal..

[12]  Brenda D. Smith,et al.  Client-service matching in substance abuse treatment for women with children. , 2002, Journal of substance abuse treatment.

[13]  L. S. Shapley,et al.  College Admissions and the Stability of Marriage , 2013, Am. Math. Mon..

[14]  B. Liang,et al.  Review of tissue plasminogen activator, ischemic stroke, and potential legal issues. , 2008, Archives of neurology.

[15]  Athanasios Drigas,et al.  An expert system for job matching of the unemployed , 2004, Expert Syst. Appl..

[16]  D G Altman,et al.  Statistics notes: Matching , 1994 .

[17]  Amitrajeet A. Batabyal,et al.  To match or not to match: Aspects of marital matchmaking under uncertainty , 2008, Oper. Res. Lett..

[18]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[19]  Douglas G Altman,et al.  Treatment allocation in controlled trials: why randomise? , 1999, BMJ.

[20]  Joseph P. Broderick,et al.  Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. , 1995 .

[21]  Jeffrey H Silber,et al.  Optimal multivariate matching before randomization. , 2004, Biostatistics.

[22]  J. Gower.  A General Coefficient of Similarity and Some of Its Properties , 1971 .

[23]  Koroshetz Wj,et al.  Tissue plasminogen activator for acute ischemic stroke. , 1996, The New England journal of medicine.

[24]  J. Cunningham,et al.  Defacto client-treatment matching: how clinicians make referrals to outpatient treatments for substance use , 2000 .

[25]  Jasjeet S. Sekhon,et al.  Genetic Optimization Using Derivatives , 2011, Political Analysis.

[26]  T. Stewart.  A CRITICAL SURVEY ON THE STATUS OF MULTIPLE CRITERIA DECISION MAKING THEORY AND PRACTICE , 1992 .

[27]  Boyan Jovanovic.  Job Matching and the Theory of Turnover , 1979, Journal of Political Economy.

[28]  Paul R. Rosenbaum,et al.  A Note on Optimal Matching With Variable Controls Using the Assignment Algorithm , 2001 .

[29]  S. Walter,et al.  Number needed to treat (NNT): estimation of a measure of clinical benefit , 2001, Statistics in medicine.

[30]  Paul R. Rosenbaum,et al.  Optimal Matching for Observational Studies , 1989 .

[31]  J. Saver,et al.  Number needed to treat estimates incorporating effects over the entire range of clinical outcomes: novel derivation method and application to thrombolytic therapy for acute stroke. , 2004, Archives of neurology.

[32]  P. Rosenbaum,et al.  Matching and thick description in an observational study of mortality after surgery. , 2001, Biostatistics.

[33]  Geoff Cohen,et al.  The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial , 2012, The Lancet.

[34]  E. Lerner.  Relationship between matching and assignment problems , 2011 .

[35]  Paul R Rosenbaum,et al.  Combining propensity score matching and group-based trajectory analysis in an observational study. , 2007, Psychological methods.

[36]  Keith Muir,et al.  Effects of alteplase beyond 3 h after stroke in the Echoplanar Imaging Thrombolytic Evaluation Trial (EPITHET): a placebo-controlled randomised trial , 2008, The Lancet Neurology.

[37]  Til Stürmer,et al.  A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. , 2006, Journal of clinical epidemiology.

[38]  Hua Li,et al.  A research on two-sided matching algorithm between new hired knowledge staff and position requirements , 2011, 2011 2nd International Conference on Artificial Intelligence, Management Science and Electronic Commerce (AIMSEC).

[39]  S. Pocock,et al.  Calculation of Numbers-Needed-To-Treat in Parallel Group Trials Assessing Ordinal Outcomes: Case Examples from Acute Stroke and Stroke Prevention , 2011, International journal of stroke : official journal of the International Stroke Society.

[40]  S. Casswell,et al.  What is meant by intention to treat analysis? Survey of published randomised controlled trials , 1999 .

[41]  Elaine L. Zanutto,et al.  Matching With Doses in an Observational Study of a Media Campaign Against Drug Abuse , 2001, Journal of the American Statistical Association.

[42]  S. Kiechl,et al.  Functional recovery after ischemic stroke—A matter of age , 2012, Neurology.