Alternative indicators for the risk of non-response bias: a simulation study.

The growth of nonresponse rates for social science surveys has led to increased concern about the risk of nonresponse bias. Unfortunately, the nonresponse rate is a poor indicator of when nonresponse bias is likely to occur. We consider in this paper a set of alternative indicators. A large-scale simulation study is used to explore how each of these indicators performs in a variety of circumstances. Although, as expected, none of the indicators fully depicts the impact of nonresponse in survey esti mates, we discuss how they can be used when creating a plausible account of the risks for nonresponse bias for a survey. We also describe an interesting characteristic of the FMI that may be helpful in diagnosing NMAR mechanisms in certain situations.

[1]  J. Graham,et al.  How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory , 2007, Prevention Science.

[2]  R. Groves.  Nonresponse Rates and Nonresponse Bias in Household Surveys , 2006 .

[3]  Frauke Kreuter,et al.  Multiple Auxiliary Variables in Nonresponse Adjustment , 2011 .

[4]  Stef van Buuren,et al.  MICE: Multivariate Imputation by Chained Equations in R , 2011 .

[5]  James Wagner,et al.  A Comparison of Alternative Indicators for the Risk of Nonresponse Bias. , 2012, Public opinion quarterly.

[6]  Geert Loosveldt,et al.  Should high response rates really be a primary objective , 2012 .

[7]  Diane K. Willimack,et al.  Response Rates and Nonresponse in BLS and Census Bureau Establishment Surveys , 2002 .

[8]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[9]  N. Nagelkerke,et al.  A note on a general definition of the coefficient of determination , 1991 .

[10]  Robert M. Groves,et al.  Responsive design for household surveys: tools for actively controlling survey errors and costs , 2006 .

[11]  Stanley Presser,et al.  Changes in Telephone Survey Nonresponse over the Past Quarter Century , 2005 .

[12]  Paul P. Biemer,et al.  Introduction to Survey Quality , 2003 .

[13]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[14]  Robert M. Groves,et al.  The Impact of Nonresponse Rates on Nonresponse Bias A Meta-Analysis , 2008 .

[15]  Thomas Lumley,et al.  Analysis of Complex Survey Samples , 2004 .

[16]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data: Little/Statistical Analysis with Missing Data , 2002 .

[17]  E. D. de Leeuw,et al.  Trends in household survey nonresponse: a longitudinal and international comparison , 2002 .

[18]  Michael A. Dimock,et al.  Gauging the Impact of Growing Nonresponse on Estimates from a National RDD Telephone Survey , 2006 .

[19]  J. Brick,et al.  Explaining Rising Nonresponse Rates in Cross-Sectional Surveys , 2013 .

[20]  Robert M. Groves,et al.  Using variation in response rates of demographic subgroups as evidence of nonresponse bias in survey estimates , 2009 .

[21]  Roderick J. A. Little,et al.  Proxy Pattern-Mixture Analysis for Survey Nonresponse , 2011 .

[22]  Jelke Bethlehem,et al.  Indicators for the representativeness of survey response , 2009 .

[23]  J. Wagner.  The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data , 2010 .

[24]  R. Little,et al.  Does Weighting for Nonresponse Increase the Variance of Survey Means? (Conference Paper) , 2004 .

[25]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .