Making sense of sensitivity: extending omitted variable bias

We extend the omitted variable bias framework with a suite of tools for sensitivity analysis in regression models that does not require assumptions on the functional form of the treatment assignment mechanism nor on the distribution of the unobserved confounders, naturally handles multiple confounders, possibly acting non‐linearly, exploits expert knowledge to bound sensitivity parameters and can be easily computed by using only standard regression results. In particular, we introduce two novel sensitivity measures suited for routine reporting. The robustness value describes the minimum strength of association that unobserved confounding would need to have, both with the treatment and with the outcome, to change the research conclusions. The partial R2 of the treatment with the outcome shows how strongly confounders explaining all the residual outcome variation would have to be associated with the treatment to eliminate the estimated effect. Next, we offer graphical tools for elaborating on problematic confounders, examining the sensitivity of point estimates and t‐values, as well as ‘extreme scenarios’. Finally, we describe problems with a common ‘benchmarking’ practice and introduce a novel procedure to bound the strength of confounders formally on the basis of a comparison with observed covariates. We apply these methods to a running example that estimates the effect of exposure to violence on attitudes toward peace.

[1]  Alex de Waal,et al.  Darfur: A New History of a Long War , 2006 .

[2]  Alexander D'Amour,et al.  Flexible Sensitivity Analysis for Observational Studies Without Observable Implications , 2018, Journal of the American Statistical Association.

[3]  Masataka Harada,et al.  A flexible, interpretable framework for assessing sensitivity to unmeasured confounding , 2016, Statistics in medicine.

[4]  J. Angrist,et al.  Undergraduate Econometrics Instruction: Through Our Classes, Darkly , 2017, SSRN Electronic Journal.

[5]  J. Robins,et al.  Sensitivity Analyses for Unmeasured Confounding Assuming a Marginal Structural Model for Repeated Measures , 2022 .

[6]  J. Pearl,et al.  Exogeneity and Robustness , 2015 .

[7]  Xu Qin,et al.  Weighting-Based Sensitivity Analysis in Causal Mediation Studies , 2018 .

[8]  M. Lovell A Simple Proof of the FWL Theorem , 2008 .

[9]  E. Oster Unobservable Selection and Coefficient Stability: Theory and Evidence , 2019 .

[10]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[11]  L. Keele,et al.  Identification, Inference and Sensitivity Analysis for Causal Mediation Effects , 2010, 1011.1079.

[12]  Paul W. Holland,et al.  The sensitivity of linear regression coefficients' confidence limits to the omission of a confounder , 2009, 0905.3463.

[13]  Christopher R. Taber,et al.  An Evaluation of Instrumental Variable Strategies for Estimating the Effects of Catholic Schooling , 2002, The Journal of Human Resources.

[14]  K. Frank,et al.  Does NBPTS Certification Affect the Number of Colleagues a Teacher Helps With Instructional Matters? , 2008 .

[15]  Jennifer Hill,et al.  Bias Amplification and Bias Unmasking , 2016, Political Analysis.

[16]  Judea Pearl,et al.  On a Class of Bias-Amplifying Variables that Endanger Effect Estimates , 2010, UAI.

[17]  M. Blackwell A Selection Bias Approach to Sensitivity Analysis for Causal Effects , 2014, Political Analysis.

[18]  R. Frisch,et al.  Partial Time Regressions as Compared with Individual Trends , 1933 .

[19]  Edward E. Leamer,et al.  S-values: Conventional context-minimal measures of the sturdiness of regression coefficients , 2016 .

[20]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[21]  E. C. Hammond,et al.  Smoking and lung cancer: recent evidence and a discussion of some questions. , 1959, Journal of the National Cancer Institute.

[22]  K. Frank,et al.  What Would It Take to Change an Inference? Using Rubin’s Causal Model to Interpret the Robustness of Causal Inferences , 2013 .

[23]  M. Lovell Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis , 1963 .

[24]  E. Oster,et al.  Unobservable Selection and Coefficient Stability: Theory and Validation , 2013 .

[25]  J. Pearl Comment on Ding and Miratrix: “To Adjust or Not to Adjust?” , 2015 .

[26]  P. Rosenbaum The Consequences of Adjustment for a Concomitant Variable that Has Been Affected by the Treatment , 1984 .

[27]  K. Frank Impact of a Confounding Variable on a Regression Coefficient , 2000 .

[28]  J. Kmenta Mostly Harmless Econometrics: An Empiricist's Companion , 2010 .

[29]  G. W. Imbens Sensitivity to Exogeneity Assumptions in Program Evaluation , 2003 .

[30]  Tyler J. VanderWeele,et al.  Sensitivity Analysis in Observational Research: Introducing the E-Value , 2017, Annals of Internal Medicine.

[31]  Onyebuchi A Arah,et al.  Bias Formulas for Sensitivity Analysis of Unmeasured Confounding for General Outcomes, Treatments, and Confounders , 2011, Epidemiology.

[32]  Thad Dunning,et al.  Natural Experiments in the Social Sciences , 2012 .

[33]  Luke W. Miratrix,et al.  To Adjust or Not to Adjust? Sensitivity Analysis of M-Bias and Butterfly-Bias , 2014, 1408.0324.

[34]  T. Shakespeare,et al.  Observational Studies , 2003 .

[35]  James M. Robins,et al.  Association, Causation, And Marginal Structural Models , 1999, Synthese.

[36]  C. Hazlett Angry or Weary? The eect of personal violence on attitudes towards peace in Darfur , 2012 .

[37]  Edward E. Leamer,et al.  Let's Take the Con Out of Econometrics , 1983 .

[38]  J. Pearl Invited commentary: understanding bias amplification. , 2011, American journal of epidemiology.

[39]  Kenneth A. Frank,et al.  10. Indices of Robustness for Sample Representation , 2007 .