Struggles with survey weighting and regression modeling

The general principles of Bayesian data analysis imply that models for survey responses should be constructed conditional on all variables that affect the probability of inclusion and nonresponse, which are also the variables used in survey weighting and clustering. However, such models can quickly become very complicated, with potentially thousands of poststratification cells. It is then a challenge to develop general families of multilevel probability models that yield reasonable Bayesian inferences. We discuss in the context of several ongoing public health and social surveys. This work is currently open-ended, and we conclude with thoughts on how research could proceed to solve these problems.

[1]  W. Deming,et al.  On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known , 1940 .

[2]  Andrew Gelman,et al.  Improving on Probability Weighting for Household Size , 1998 .

[3]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[4]  A. Brix.  Bayesian Data Analysis, 2nd edn , 2005 .

[5]  D. Pfeffermann.  The Role of Sampling Weights when Modeling Survey Data , 1993 .

[6]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[7]  Carl-Erik Särndal,et al.  Generalized Raking Procedures in Survey Sampling , 1993 .

[8]  D. Binder.  On the variances of asymptotically normal estimators from complex surveys , 1983 .

[9]  Robert D. Tortora,et al.  Sampling: Design and Analysis , 2000 .

[10]  R. Little,et al.  Random-effects Models for Smoothing Poststrati ® cation Weights , 1999 .

[11]  Gary King,et al.  THE POLLS-A REVIEW PREELECTION SURVEY METHODOLOGY: DETAILS FROM EIGHT POLLING ORGANIZATIONS, 1988 AND 1992 , 1995 .

[12]  Andrew Gelman,et al.  Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls , 2004, Political Analysis.

[13]  N. Garmezy,et al.  Vulnerability and resilience. , 1993 .

[14]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[15]  Gary King,et al.  Pre-Election Survey Methodology: Details from Nine Polling Organizations, 1988 and 1992 , 2008 .

[16]  R. Fay,et al.  Estimates of Income for Small Places: An Application of James-Stein Procedures to Census Data , 1979 .

[17]  J. Carlin,et al.  Poststratification and Weighting Adjustments , 2000 .

[18]  Andrew Gelman,et al.  Survey Weighting and Regression , 2005 .

[19]  Andrew Gelman,et al.  A method for estimating design-based sampling variances for surveys with weighting, poststratification, and , 2003 .

[20]  E. Korn,et al.  Inference for Superpopulation Parameters Using Sample Surveys , 2002 .

[21]  W. DuMouchel,et al.  Using Sample Survey Weights in Multiple Regression Analyses of Stratified Samples , 1983 .

[22]  R. Little.  Post-Stratification: A Modeler's Perspective , 1993 .

[23]  R. Little,et al.  Model-Based Alternatives to Trimming Survey Weights , 2000 .