The Balance-Sample Size Frontier in Matching Methods for Causal Inference

We propose a simplified approach to matching for causal inference that simultaneously optimizes balance (similarity between the treated and control groups) and matched sample size. Existing approaches either fix the matched sample size and maximize balance or fix balance and maximize sample size, leaving analysts to settle for suboptimal solutions or attempt manual optimization by iteratively tweaking their matching method and rechecking balance. To jointly maximize balance and sample size, we introduce the matching frontier, the set of matching solutions with maximum possible balance for each sample size. Rather than iterating, researchers can choose matching solutions from the frontier for analysis in one step. We derive fast algorithms that calculate the matching frontier for several commonly used balance metrics. We demonstrate this approach with analyses of the effect of sex on judging and job training programs that show how the methods we introduce can extract new knowledge from existing data sets.

[1]  I. Shpitser,et al.  A New Criterion for Confounder Selection , 2011, Biometrics.

[2]  Gary King,et al.  Misunderstandings between experimentalists and observationalists about causal inference , 2008 .

[3]  Gary King,et al.  The Dangers of Extreme Counterfactuals , 2006, Political Analysis.

[4]  M. Sobel,et al.  Identification Problems in the Social Sciences , 1996 .

[5]  C. Blumberg.  Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction , 2016 .

[6]  Peter C Austin,et al.  A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003 , 2008, Statistics in medicine.

[7]  Donald B Rubin,et al.  On the limitations of comparative effectiveness research , 2010, Statistics in medicine.

[8]  Jasjeet S. Sekhon,et al.  Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching Package for R , 2008 .

[9]  Marco Caliendo,et al.  Some Practical Guidance for the Implementation of Propensity Score Matching , 2005, SSRN Electronic Journal.

[10]  Andrew D. Martin,et al.  Untangling the Causal Effects of Sex on Judging , 2010 .

[11]  Burt S. Barnow,et al.  Issues in the Analysis of Selectivity Bias. Discussion Papers. Revised. , 1980 .

[12]  D. Rubin.  For objective causal inference, design trumps analysis , 2008, 0811.1640.

[13]  B. Shepherd,et al.  GUIDO IMBENS, DONALD RUBIN, Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. New York: Cambridge University Press. , 2016, Biometrics.

[14]  J. Pearl,et al.  Causal inference , 2011, Twenty-one Mental Models That Can Change Policing.

[15]  A. Hung,et al.  Reweighted Mahalanobis distance matching for cluster‐randomized trials with missing data , 2012, Pharmacoepidemiology and drug safety.

[16]  Elizabeth A Stuart,et al.  Developing practical recommendations for the use of propensity scores: Discussion of ‘A critical appraisal of propensity score matching in the medical literature between 1996 and 2003’ by Peter Austin, Statistics in Medicine , 2008, Statistics in medicine.

[17]  Zhong Zhao,et al.  Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence , 2004, Review of Economics and Statistics.

[18]  Judea Pearl,et al.  Causal Inference , 2010 .

[19]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[20]  Susan Athey,et al.  A Measure of Robustness to Misspecification , 2015 .

[21]  G. King,et al.  Multivariate Matching Methods That Are Monotonic Imbalance Bounding , 2011 .

[22]  Richard A. Nielsen,et al.  MatchingFrontier : Automated Matching for Causal Inference ∗ , 2015 .

[23]  E. Stuart,et al.  Misunderstandings among Experimentalists and Observationalists about Causal Inference , 2007 .

[24]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[25]  R. Lalonde.  Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[26]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[27]  P. Rosenbaum,et al.  Minimum Distance Matched Sampling With Fine Balance in an Observational Study of Treatment for Ovarian Cancer , 2007 .

[28]  G. King,et al.  Causal Inference without Balance Checking: Coarsened Exact Matching , 2012, Political Analysis.

[29]  Virginia A. Hettinger,et al.  Picking Federal Judges: A Note on Policy and Partisan Selection Agendas , 2001 .

[30]  C. Glymour,et al.  STATISTICS AND CAUSAL INFERENCE , 1985 .

[31]  Andrew D. Martin,et al.  The Judicial Common Space , 2007 .

[32]  Tyler J VanderWeele,et al.  Causal inference under multiple versions of treatment , 2013, Journal of causal inference.

[33]  Richard K. Crump,et al.  Dealing with limited overlap in estimation of average treatment effects , 2009 .

[34]  J. Sekhon,et al.  Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies , 2006, Review of Economics and Statistics.

[35]  Gary King,et al.  Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference , 2007, Political Analysis.

[36]  Rajeev Dehejia,et al.  Propensity Score-Matching Methods for Nonexperimental Causal Studies , 2002, Review of Economics and Statistics.