Deep Generalized Method of Moments for Instrumental Variable Analysis

Instrumental variable analysis is a powerful tool for estimating causal effects when randomization or full control of confounders is not possible. The application of standard methods such as 2SLS, GMM, and more recent variants are significantly impeded when the causal effects are complex, the instruments are high-dimensional, and/or the treatment is high-dimensional. In this paper, we propose the DeepGMM algorithm to overcome this. Our algorithm is based on a new variational reformulation of GMM with optimal inverse-covariance weighting that allows us to efficiently control very many moment conditions. We further develop practical techniques for optimization and model selection that make it particularly successful in practice. Our algorithm is also computationally tractable and can handle large-scale datasets. Numerical results show our algorithm matches the performance of the best tuned methods in standard settings and continues to work in high-dimensional settings where even recent methods break.

[1]  T. Sargent,et al.  FORMULATING AND ESTIMATING DYNAMIC LINEAR RATIONAL EXPECTATIONS MODELS , 1980 .

[2]  A. Walker,et al.  Drug Copayment and Adherence in Chronic Heart Failure: Effect on Cost and Outcomes , 2006, Pharmacotherapy.

[3]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[4]  D. Rubin.  Matched Sampling for Causal Effects: Matching to Remove Bias in Observational Studies , 1973 .

[5]  Xiaohong Chen,et al.  Semi‐Nonparametric IV Estimation of Shape‐Invariant Engel Curves , 2003 .

[6]  Joshua D. Angrist,et al.  Mostly Harmless Econometrics: An Empiricist's Companion , 2008 .

[7]  Nathan Kallus,et al.  DeepMatch: Balancing Deep Covariate Representations for Causal Inference Using Adversarial Training , 2018, ICML.

[8]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[9]  D. Pollard.  Empirical Processes: Theory and Applications , 1990 .

[10]  W. Newey,et al.  Instrumental variable estimation of nonparametric models , 2003 .

[11]  Steven T. Berry,et al.  Automobile Prices in Market Equilibrium , 1995 .

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  G. Chamberlain.  Asymptotic efficiency in estimation with conditional moment restrictions , 1987 .

[14]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[15]  Xiaohong Chen,et al.  Optimal Sup-Norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression , 2015, 1508.03365.

[16]  Constantinos Daskalakis,et al.  Training GANs with Optimism , 2017, ICLR.

[17]  Uri Shalit,et al.  Estimating individual treatment effect: generalization bounds and algorithms , 2016, ICML.

[18]  J. Angrist,et al.  Journal of Economic Perspectives—Volume 15, Number 4—Fall 2001—Pages 69–85 Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments , 2022 .

[19]  J. Florens,et al.  Nonparametric Instrumental Regression , 2010 .

[20]  Nathan Kallus,et al.  Generalized Optimal Matching Methods for Causal Inference , 2016, J. Mach. Learn. Res..

[21]  Uri Shalit,et al.  Learning Representations for Counterfactual Inference , 2016, ICML.

[22]  Xiaohong Chen,et al.  Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions , 2003 .

[23]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[24]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[25]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[26]  Fredrik D. Johansson,et al.  Learning Weighted Representations for Generalization Across Designs , 2018, 1802.08598.

[27]  Vasilis Syrgkanis,et al.  Adversarial Generalized Method of Moments , 2018, ArXiv.

[28]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[29]  Kevin Leyton-Brown,et al.  Deep IV: A Flexible Approach for Counterfactual Prediction , 2017, ICML.

[30]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[32]  L. Hansen.  Large Sample Properties of Generalized Method of Moments Estimators , 1982 .

[33]  Peter L. Bartlett,et al.  Nearly-tight VC-dimension and Pseudodimension Bounds for Piecewise Linear Neural Networks , 2017, J. Mach. Learn. Res..

[34]  J. Geanakoplos,et al.  Generalized Instrumental Variables Estimation of Nonlinear Rational Expectations Models , 2007 .

[35]  Oriol Vinyals,et al.  Learning Implicit Generative Models with the Method of Learned Moments , 2018, ICML.

[36]  L. Hansen,et al.  Finite Sample Properties of Some Alternative Gmm Estimators , 2015 .

[37]  Xiaohong Chen,et al.  Estimation of Nonparametric Conditional Moment Models with Possibly Nonsmooth Generalized Residuals , 2009 .