Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies

Abstract This paper presents genetic matching, a method of multivariate matching that uses an evolutionary search algorithm to determine the weight each covariate is given. Both propensity score matching and matching based on Mahalanobis distance are limiting cases of this method. The algorithm makes transparent certain issues that all matching methods must confront. We present simulation studies that show that the algorithm improves covariate balance and that it may reduce bias if the selection on observables assumption holds. We then present a reanalysis of a number of data sets in the LaLonde (1986) controversy.

[1]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[2]  D. Horvitz,et al.  A Generalization of Sampling Without Replacement from a Finite Universe , 1952 .

[3]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[4]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[5]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[6]  W. G. Cochran,et al.  Controlling Bias in Observational Studies: A Review. , 1974 .

[7]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[8]  Donald B. Rubin,et al.  Multivariate matching methods that are equal percent bias reducing , 1974 .

[9]  Donald B. Rubin,et al.  MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING, I: SOME EXAMPLES , 1974 .

[10]  O. Ashenfelter,et al.  Estimating the Effect of Training Programs on Earnings , 1976 .

[11]  D. Rubin.  Assignment to Treatment Group on the Basis of a Covariate , 1976 .

[12]  Donald B. Rubin,et al.  Bayesian Inference for Causal Effects: The Role of Randomization , 1978 .

[13]  P. Billingsley,et al.  Probability and Measure , 1980 .

[14]  R. Hogg.  An Introduction to Robust Estimation , 1979 .

[15]  A. Dawid.  Conditional Independence in Statistical Theory , 1979 .

[16]  David F. Hendry,et al.  Econometrics-Alchemy or Science? , 1980 .

[17]  D. Rubin.  Bias Reduction Using Mahalanobis-Metric Matching , 1980 .

[18]  Edward E. Leamer,et al.  Let's Take the Con Out of Econometrics , 1983 .

[19]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[20]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[21]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[22]  Orley Ashenfelter,et al.  Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs , 1984 .

[23]  R. Lalonde.  Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[24]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[25]  V. Cerný.  Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm , 1985 .

[26]  A. F. Mitchell,et al.  The Mahalanobis distance and elliptic distributions , 1985 .

[27]  Patrick Billingsley,et al.  Probability and Measure. , 1986 .

[28]  P. Holland.  Statistics and Causal Inference , 1985 .

[29]  David Card,et al.  Measuring the Effect of Subsidized Training Programs on Movements in Andout of Employment , 1987 .

[30]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[31]  Amendments and Corrections: The Mahalanobis Distance and Elliptic Distributions , 1989 .

[32]  James J. Heckman,et al.  Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: the Case of Manpower Training , 1989 .

[33]  Robert E. Schapire.  The Strength of Weak Learnability , 1989, COLT 1989.

[34]  John J. Grefenstette,et al.  How Genetic Algorithms Work: A Critical Look at Implicit Parallelism , 1989, ICGA.

[35]  Paul R. Rosenbaum,et al.  Optimal Matching for Observational Studies , 1989 .

[36]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[37]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[38]  D. Rubin.  [On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.] Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies , 1990 .

[39]  P. Rosenbaum.  A Characterization of Optimal Designs for Observational Studies , 1991 .

[40]  Donald B. Rubin,et al.  Characterizing the effect of matching using linear propensity score methods with normal distributions , 1992 .

[41]  P. Hall.  The Bootstrap and Edgeworth Expansion , 1992 .

[42]  Michael D. Vose,et al.  Modeling Simple Genetic Algorithms , 1992, FOGA.

[43]  Donald B. Rubin,et al.  Affinely Invariant Matching Methods with Ellipsoidal Distributions , 1992 .

[44]  Kenneth A. De Jong,et al.  Genetic Algorithms are NOT Function Optimizers , 1992, FOGA.

[45]  John J. Grefenstette,et al.  Deception Considered Harmful , 1992, FOGA.

[46]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[47]  C. Drake.  Effects of misspecification of the propensity score on estimators of treatment effect , 1993 .

[48]  Cesare Alippi,et al.  Genetic-algorithm programming environments , 1994, Computer.

[49]  J. Robins,et al.  Estimation of Regression Coefficients When Some Regressors are not Always Observed , 1994 .

[50]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[51]  J. Robins,et al.  Analysis of semiparametric regression models for repeated outcomes in the presence of missing data , 1995 .

[52]  Donald Rubin,et al.  Estimating Causal Effects from Large Data Sets Using Propensity Scores , 1997, Annals of Internal Medicine.

[53]  Herbert L. Smith.  6. Matching with Multiple Controls to Estimate Treatment Effects in Observational Studies , 1997 .

[54]  J. Hahn.  On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects , 1998 .

[55]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[56]  J. Sekhon,et al.  Genetic Optimization Using Derivatives , 1998, Political Analysis.

[57]  Christopher Winship,et al.  THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA , 1999 .

[58]  T. DiPrete,et al.  Estimating Causal Effects With Matching Methods in the Presence and Absence of Bias Cancellation , 2000 .

[59]  P. Bühlmann,et al.  Boosting with the L2-loss: regression and classification , 2001 .

[60]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[61]  Petra E. Todd,et al.  Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods , 2001 .

[62]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[63]  Rajeev Dehejia,et al.  Propensity Score-Matching Methods for Nonexperimental Causal Studies , 2002, Review of Economics and Statistics.

[64]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[65]  Alberto Abadie.  Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models , 2002 .

[66]  Donald B. Rubin,et al.  Principal Stratification Approach to Broken Randomized Experiments , 2003 .

[67]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[68]  Guido W. Imbens,et al.  EFFICIENT ESTIMATION OF AVERAGE TREATMENT EFFECTS , 2003 .

[69]  N. Christakis,et al.  The health impact of health care on families: a matched cohort study of hospice use by decedents and mortality outcomes in surviving, widowed spouses. , 2003, Social science & medicine.

[70]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[71]  B. Hansen.  Full Matching in an Observational Study of Coaching for the SAT , 2004 .

[72]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[73]  G. Imbens,et al.  Large Sample Properties of Matching Estimators for Average Treatment Effects , 2004 .

[74]  Sebastian Galiani,et al.  Water for Life: The Impact of the Privatization of Water Services on Child Mortality , 2004, Journal of Political Economy.

[75]  D. Rubin.  Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation , 2001, Health Services and Outcomes Research Methodology.

[76]  Steven F. Lehrer,et al.  Matching using semiparametric propensity scores , 2013 .

[77]  Jasjeet S. Sekhon,et al.  Quality Meets Quantity: Case Studies, Conditional Probability, and Counterfactuals , 2004, Perspectives on Politics.

[78]  Sergio Firpo.  Efficient Semiparametric Estimation of Quantile Treatment Effects , 2004 .

[79]  D. McCaffrey,et al.  Propensity score estimation with boosted regression for evaluating causal effects in observational studies. , 2004, Psychological methods.

[80]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[81]  Michael D. Vose,et al.  Modeling genetic algorithms with Markov chains , 1992, Annals of Mathematics and Artificial Intelligence.

[82]  John C. Ham,et al.  Special issue on Experimental and non-experimental evaluation of economic policy and models , 2005 .

[83]  Kosuke Imai,et al.  Do Get-Out-the-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments , 2005, American Political Science Review.

[84]  Rajeev Dehejia.  Practical propensity score matching: a reply to Smith and Todd , 2005 .

[85]  B. Hansen,et al.  Optimal Full Matching and Related Designs via Network Flows , 2006 .

[86]  Donald B. Rubin,et al.  Affinely invariant matching methods with discriminant mixtures of proportional ellipsoidally symmetric distributions , 2006, math/0611263.

[87]  Richard K. Crump,et al.  Moving the Goalposts: Addressing Limited Overlap in Estimation of Average Treatment Effects by Changing the Estimand , 2006, SSRN Electronic Journal.

[88]  Joseph Kang,et al.  Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data , 2007, 0804.2958.

[89]  D. Rubin.  Matched Sampling for Causal Effects , 2006 .

[90]  Matching Estimators of Causal Effects , 2006 .

[91]  E. Stuart,et al.  Misunderstandings among Experimentalists and Observationalists about Causal Inference , 2007 .

[92]  Jonathan N. Wand,et al.  Assessing partisan bias in voting technology: The case of the 2004 New Hampshire recount , 2007 .

[93]  G. Huber,et al.  The Effect of Electoral Competitiveness on Incumbent Behavior , 2007 .

[94]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[95]  Henry E. Brady,et al.  The Varying Role of Voter Information Across Democratic Societies , 2007 .

[96]  Gary King,et al.  Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference , 2007, Political Analysis.

[97]  Jay Bhattacharya,et al.  Do Instrumental Variables Belong in Propensity Scores? , 2007 .

[98]  Jake Bowers,et al.  Covariate balance in simple stratified and clustered comparative studies , 2008, 0808.3857.

[99]  D. Freedman,et al.  Weighting Regressions by Propensity Scores , 2008, Evaluation review.

[100]  Jasjeet S. Sekhon,et al.  Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching Package for R , 2008 .

[101]  Jasjeet S. Sekhon,et al.  A New Non-Parametric Matching Method for Bias Adjustment with Applications to Economic Evaluations , 2008 .

[102]  Richard Grieve,et al.  Evaluating health care programs by combining cost with quality of life measures: a case study comparing capitation and fee for service. , 2008, Health services research.

[103]  Gary King,et al.  The Supreme Court During Crisis: How War Affects Only Non-War Cases , 2008 .

[104]  Peter C Austin,et al.  A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003 , 2008, Statistics in medicine.

[105]  Gary King,et al.  Misunderstandings between experimentalists and observationalists about causal inference , 2008 .

[106]  S. Schneeweiss,et al.  Evaluating uses of data mining techniques in propensity score estimation: a simulation study , 2008, Pharmacoepidemiology and drug safety.

[107]  D. Rubin.  For objective causal inference, design trumps analysis , 2008, 0811.1640.

[108]  P. Austin.  Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples , 2009, Statistics in medicine.

[109]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[110]  Elizabeth A Stuart,et al.  Improving propensity score weighting using machine learning , 2010, Statistics in medicine.

[111]  Judea Pearl,et al.  On a Class of Bias-Amplifying Variables that Endanger Effect Estimates , 2010, UAI.

[112]  Gary King,et al.  MatchIt: Nonparametric Preprocessing for Parametric Causal Inference , 2011 .

[113]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[114]  Bryan S. Graham,et al.  Efficient Estimation of Data Combination Models by the Method of Auxiliary-to-Study Tilting (AST) , 2011 .

[115]  Kristin E. Porter,et al.  The Relative Performance of Targeted Maximum Likelihood Estimators , 2011, The international journal of biostatistics.

[116]  Jasjeet S. Sekhon,et al.  Genetic Optimization Using Derivatives , 2011, Political Analysis.

[117]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[118]  Jens Hainmueller,et al.  Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies , 2012, Political Analysis.