Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing

Inverse propensity weighting (IPW) is a popular method for estimating treatment effects from observational data. However, its correctness relies on the untestable (and frequently implausible) assumption that all confounders have been measured. This paper introduces a robust sensitivity analysis for IPW that estimates the range of treatment effects compatible with a given amount of unobserved confounding. The estimated range converges to the narrowest possible interval (under the given assumptions) that must contain the true treatment effect. Our proposal is a refinement of the influential sensitivity analysis by Zhao, Small, and Bhattacharya (2019), which we show gives bounds that are too wide even asymptotically. This analysis is based on new partial identification results for Tan (2006)’s marginal sensitivity model.

[1]  G. Dantzig,et al.  On the Fundamental Lemma of Neyman and Pearson , 1951 .

[2]  G. Johnson.  Economic Analysis of Trade Unionism , 1975 .

[3]  K. C. G. Chan,et al.  Globally efficient non‐parametric inference of average treatment effects by empirical balancing calibration weighting , 2016, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[4]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[5]  Richard B. Freeman,et al.  Longitudinal Analyses of the Effects of Trade Unions , 1983, Journal of Labor Economics.

[6]  Victor Chernozhukov,et al.  Conditional Quantile Processes Based on Series or Many Regressors , 2011, Journal of Econometrics.

[7]  A. Schick.  On Asymptotically Efficient Estimation in Semiparametric Models , 1986 .

[8]  Alexandre Poirier,et al.  Assessing Sensitivity to Unconfoundedness: Estimation and Inference , 2020, Journal of Business & Economic Statistics.

[9]  Zhiqiang Tan,et al.  A Distributional Approach for Causal Inference Using Propensity Scores , 2006 .

[10]  Xiaojie Mao,et al.  Interval Estimation of Individual-Level Causal Effects Under Unobserved Confounding , 2018, AISTATS.

[11]  E. Stuart,et al.  Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies , 2015, Statistics in medicine.

[12]  D. Darling.  On a Class of Problems Related to the Random Division of an Interval , 1953 .

[13]  S. Kruger.  Design Of Observational Studies , 2016 .

[14]  George H. Jakubson.  Estimation and Testing of the Union Wage Effect Using Panel Data , 1991 .

[15]  Stefan Wager,et al.  Policy Learning With Observational Data , 2017, Econometrica.

[16]  Nathan Kallus,et al.  Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learning , 2020, NeurIPS.

[17]  Naomi S. Altman,et al.  Quantile regression , 2019, Nature Methods.

[18]  W. Mellow.  Unionism and Wages: A Longitudinal Analysis , 1981 .

[19]  Causal Rule Ensemble: Interpretable Inference of Heterogeneous Treatment Effects , 2020, 2009.09036.

[20]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[21]  P. Rosenbaum.  Sensitivity analysis for certain permutation inferences in matched observational studies , 1987 .

[22]  Nicolai Meinshausen,et al.  Quantile Regression Forests , 2006, J. Mach. Learn. Res..

[23]  G. Imbens,et al.  Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score , 2000 .

[24]  J. Robins,et al.  Sensitivity Analysis for Selection bias and unmeasured Confounding in missing Data and Causal inference models , 2000 .

[25]  K. Imai,et al.  Covariate balancing propensity score , 2014 .

[26]  S. Athey,et al.  Generalized random forests , 2016, The Annals of Statistics.

[27]  Nathan Kallus,et al.  Minimax-Optimal Policy Learning Under Unobserved Confounding , 2020, Manag. Sci..

[28]  Dylan S. Small,et al.  Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap , 2017, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[29]  I. Molchanov,et al.  Sharp identification regions in models with convex moment predictions , 2010 .

[30]  John Duchi,et al.  BOUNDS ON THE CONDITIONAL AND AVERAGE TREATMENT EFFECT WITH UNOBSERVED CONFOUNDING FACTORS. , 2018, Annals of statistics.

[31]  M. Kosorok.  Introduction to Empirical Processes and Semiparametric Inference , 2008 .

[32]  Stefan Wager,et al.  Shape-constrained partial identification of a population mean under unknown probabilities of sample selection , 2017, 1706.07550.

[33]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[34]  Guillaume Basse,et al.  Combining observational and experimental datasets using shrinkage estimators. , 2020, Biometrics.

[35]  Donald K. K. Lee,et al.  Interval estimation of population means under unknown but bounded probabilities of sample selection , 2013 .

[36]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[37]  Nathan Kallus,et al.  Confounding-Robust Policy Improvement , 2018, NeurIPS.

[38]  P. Rosenbaum.  Covariance Adjustment in Randomized Experiments and Observational Studies , 2002 .

[39]  Andrea Rotnitzky,et al.  Pattern–mixture and selection models for analysing longitudinal data with monotone missing patterns , 2003 .

[40]  An Interval Estimation Approach to Sample Selection Bias , 2019, 1906.10159.

[41]  G. Chamberlain.  Multivariate regression models for panel data , 1982 .

[42]  G. Imbens,et al.  Estimation of Causal Effects using Propensity Score Weighting: An Application to Data on Right Heart Catheterization , 2001, Health Services and Outcomes Research Methodology.

[43]  J. Robins,et al.  Double/Debiased Machine Learning for Treatment and Structural Parameters , 2017 .

[44]  Whitney K. Newey,et al.  Cross-fitting and fast remainder rates for semiparametric estimation , 2017, 1801.09138.