Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation

Propensity score methodology can be used to help design observational studies in a way analogous to the way randomized experiments are designed: without seeing any answers involving outcome variables. The typical models used to analyze observational data (e.g., least squares regressions, difference of difference methods) involve outcomes, and so cannot be used for design in this sense. Because the propensity score is a function only of covariates, not outcomes, repeated analyses attempting to balance covariate distributions across treatment groups do not bias estimates of the treatment effect on outcome variables. This theme will the primary focus of this article: how to use the techniques of matching, subclassification and/or weighting to help design observational studies. The article also proposes a new diagnostic table to aid in this endeavor, which is especially useful when there are many covariates under consideration. The conclusion of the initial design phase may be that the treatment and control groups are too far apart to produce reliable effect estimates without heroic modeling assumptions. In such cases, it may be wisest to abandon the intended observational study, and search for a more acceptable data set where such heroic modeling assumptions are not necessary. The ideas and techniques will be illustrated using the initial design of an observational study for use in the tobacco litigation based on the NMES data set.

[1]  Charles C. Peters,et al.  A Method of Matching Groups for Experiment with No Loss of Population , 1941 .

[2]  D. Horvitz,et al.  A Generalization of Sampling Without Replacement from a Finite Universe , 1952 .

[3]  W. G. Cochran.  Analysis of covariance: Its nature and uses. , 1957 .

[4]  D. Rubin.  Matched Sampling for Causal Effects: The Use of Matched Sampling and Regression Adjustment to Remove Bias in Observational Studies , 1973 .

[5]  D. Rubin.  Matched Sampling for Causal Effects: Matching to Remove Bias in Observational Studies , 1973 .

[6]  Donald B. Rubin,et al.  MULTIVARIATE MATCHING METHODS THAT ARE EQUAL PERCENT BIAS REDUCING, I: SOME EXAMPLES , 1974 .

[7]  W. G. Cochran,et al.  Controlling Bias in Observational Studies: A Review. , 1974 .

[8]  Donald B. Rubin,et al.  Multivariate matching methods that are equal percent bias reducing , 1974 .

[9]  O S Miettinen,et al.  Stratification by a multivariate confounder score. , 1976, American journal of epidemiology.

[10]  D. Rubin.  ASSIGNMENT TO TREATMENT GROUP ON THE BASIS OF A COVARIATE , 1976 .

[11]  J. Heckman.  The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models , 1976 .

[12]  R. G. Carpenter,et al.  Matching when covariables are normally distributed , 1977 .

[13]  R. Horwitz.  The planning of observational studies of human populations , 1979 .

[14]  D. Rubin,et al.  Using Multivariate Matched Sampling and Regression Adjustment to Control Bias in Observational Studies , 1978 .

[15]  D. Oakes,et al.  Statistical Methods for Comparative Studies , 1980 .

[16]  D. Rubin.  Bias Reduction Using Mahalanobis-Metric Matching , 1980 .

[17]  D. Rubin,et al.  Assessing Sensitivity to an Unobserved Binary Covariate in an Observational Study with Binary Outcome , 1983 .

[18]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[19]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[20]  R. Lalonde.  Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[21]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[22]  James J. Heckman,et al.  Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: the Case of Manpower Training , 1989 .

[23]  Paul R. Rosenbaum,et al.  Optimal Matching for Observational Studies , 1989 .

[24]  P. Rosenbaum.  A Characterization of Optimal Designs for Observational Studies , 1991 .

[25]  Roderick J. A. Little,et al.  Projecting From Advance Data Using Propensity Modeling: An Application to Income and Tax Statistics , 1992 .

[26]  Donald B. Rubin,et al.  Characterizing the effect of matching using linear propensity score methods with normal distributions , 1992 .

[27]  Donald B. Rubin,et al.  Affinely Invariant Matching Methods with Ellipsoidal Distributions , 1992 .

[28]  Paul R. Rosenbaum,et al.  Comparison of Multivariate Matching Methods: Structures, Distances, and Algorithms , 1993 .

[29]  David Card,et al.  Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania , 1993 .

[30]  D. Rubin,et al.  In utero exposure to phenobarbital and intelligence deficits in adult men. , 1995, JAMA.

[31]  D B Rubin,et al.  Matching using estimated propensity scores: relating theory to practice. , 1996, Biometrics.

[32]  G. Imbens.  The Role of the Propensity Score in Estimating Dose-Response Functions , 1999 .

[33]  D. Rubin,et al.  Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates , 2000 .

[34]  D. Rubin,et al.  Estimating and Using Propensity Scores with Partially Missing Data , 2000 .

[35]  Donald B. Rubin,et al.  Statistical Issues in the Estimation of the Causal Effects of Smoking Due to the Conduct of the Tobacco Industry , 2000 .

[36]  Scott L. Zeger,et al.  Statistical Testimony on Damages in Minnesota v. Tobacco Industry , 2000 .

[37]  D B Rubin,et al.  Estimating the causal effects of smoking , 2001, Statistics in medicine.

[38]  D. Rubin,et al.  Principal Stratification in Causal Inference , 2002, Biometrics.

[39]  Donald B. Rubin,et al.  Matched Sampling for Causal Effects: William G. Cochran's Contributions to the Design, Analysis, and Evaluation of Observational Studies , 2006 .