A Bayesian analysis of design parameters in survey data collection

In the design of surveys, a number of input parameters such as contact propensities, participation propensities, and costs per sample unit play a decisive role. In ongoing surveys, these survey design parameters are usually estimated from previous experience and updated gradually with new experience. In new surveys, these parameters are estimated from expert opinion and experience with similar surveys. Although survey institutes have fair expertise and experience, the postulation, estimation, and updating of survey design parameters is rarely done in a systematic way. This article presents a Bayesian framework to include and update prior knowledge and expert opinion about the parameters. This framework is set in the context of adaptive survey designs in which different population units may receive different treatment given quality and cost objectives. For this type of survey, the accuracy of design parameters becomes even more crucial to effective design decisions. The framework allows for a Bayesian analysis of the performance of a survey during data collection and in between waves of a survey. We demonstrate the utility of the Bayesian analysis using a simulation study based on the Dutch Health Survey.

[1]  Barry Schouten,et al.  Optimizing quality of response through adaptive survey designs , 2013 .

[2]  Robert M. Groves,et al.  Responsive design for household surveys: tools for actively controlling survey errors and costs , 2006 .

[3]  Andy Peytchev,et al.  Prioritizing Low Propensity Sample Members in a Survey: Implications for Nonresponse Bias , 2014 .

[4]  Gabriele B. Durrant,et al.  Analysing interviewer call record data by using a multilevel discrete-time event history modelling approach , 2013 .

[5]  Natalie Shlomo,et al.  Selecting Adaptive Survey Design Strata with Partial R-indicators , 2017 .

[6]  S. Murphy,et al.  Dynamic Treatment Regimes. , 2014, Annual review of statistics and its application.

[7]  Eric B. Laber,et al.  Dynamic treatment regimes: Technical challenges and applications , 2014 .

[8]  Andrey Peytchev,et al.  Reduction of Nonresponse Bias through Case Prioritization , 2010 .

[9]  Peter Lundquist,et al.  ACCURACY IN ESTIMATION WITH NONRESPONSE: A FUNCTION OF DEGREE OF IMBALANCE AND DEGREE OF EXPLANATION , 2014 .

[10]  Gabriele B. Durrant,et al.  Modeling Final Outcome and Length of Call Sequence to Improve Efficiency in Interviewer Call Scheduling , 2015 .

[11]  Barry Schouten,et al.  Tailored fieldwork design to increase representative household survey response: an experiment in the Survey of Consumer Satisfaction , 2013 .

[12]  J. Ibrahim,et al.  Power prior distributions for regression models , 2000 .

[13]  Michael R Elliott,et al.  Alternative indicators for the risk of non-response bias: a simulation study. , 2016, International statistical review = Revue internationale de statistique.

[14]  Jelke Bethlehem,et al.  Handbook of Nonresponse in Household Surveys: Bethlehem/Handbook of Nonresponse , 2011 .

[15]  Joseph G Ibrahim,et al.  The power prior: theory and applications , 2015, Statistics in medicine.

[16]  Gabriele B. Durrant,et al.  Using paradata to predict best times of contact, conditioning on household and interviewer influences , 2011 .

[17]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[18]  B. Carlin,et al.  Adaptive design improvements in the continual reassessment method for phase I studies. , 1999, Statistics in medicine.

[19]  J O'Quigley,et al.  Continual reassessment method: a likelihood approach. , 1996, Biometrics.

[20]  S. Murphy,et al.  Optimal dynamic treatment regimes , 2003 .

[21]  Jelke Bethlehem,et al.  Indicators for the representativeness of survey response , 2009 .

[22]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[23]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[24]  Anastasios A. Tsiatis,et al.  Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes , 2012, Statistical science : a review journal of the Institute of Mathematical Statistics.

[25]  Frauke Kreuter,et al.  Improving Surveys with Paradata: Analytic Uses of Process Information , 2013 .

[26]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[27]  Jelke Bethlehem,et al.  Handbook of Nonresponse in Household Surveys , 2011 .

[28]  S. Murphy,et al.  A "SMART" design for building individualized treatment sequences. , 2012, Annual review of clinical psychology.

[29]  Daniel O Scharfstein,et al.  Incorporating prior beliefs about selection bias into the analysis of randomized trials with missing outcomes. , 2003, Biostatistics.

[30]  James Wagner,et al.  Use of Paradata in a Responsive Design Framework to Manage a Field Data Collection , 2012 .

[31]  W. Robinson,et al.  Respondent-Driven Sampling of Heterosexuals at Increased Risk of HIV Infection. , 2014, Survey practice.