Population Survey Features and Response Rates: A Randomized Experiment.

OBJECTIVES To study the effects of several survey features on response rates in a general population health survey. METHODS In 2012 and 2013, 8000 households in British Columbia, Canada, were randomly allocated to 1 of 7 survey variants, each containing a different combination of survey features. Features compared included administration modes (paper vs online), prepaid incentive ($2 coin vs none), lottery incentive (instant vs end-of-study), questionnaire length (10 minutes vs 30 minutes), and sampling frame (InfoCanada vs Canada Post). RESULTS The overall response rate across the 7 groups was 27.9% (range = 17.1-43.4). All survey features except the sampling frame were associated with statistically significant differences in response rates. The survey mode elicited the largest effect on the odds of response (odds ratio [OR] = 2.04; 95% confidence interval [CI] = 1.61, 2.59), whereas the sampling frame showed the least effect (OR = 1.14; 95% CI = 0.98, 1.34). The highest response was achieved by mailing a short paper survey with a prepaid incentive. CONCLUSIONS In a mailed general population health survey in Canada, a 40% to 50% response rate can be expected. Questionnaire administration mode, survey length, and type of incentive affect response rates.

[1]  Sarah M. Greene,et al.  Surveying Physicians: Do Components of the “Total Design Approach” to Optimizing Survey Response Rates Apply to Physicians? , 2002, Medical care.

[2]  D. Pfeffermann.  The Role of Sampling Weights when Modeling Survey Data , 1993 .

[3]  J. Ziegenfuss,et al.  Shortening a survey and using alternative forms of prenotification: Impact on response rate and quality , 2010, BMC medical research methodology.

[4]  D. Dillman,et al.  Increasing response to personally-delivered mail-back questionnaires. , 1995 .

[5]  Tracy L. Tuten,et al.  Effects of Immediate Versus Delayed Notification of Prize Draw Results on Response Behavior in Web Surveys , 2004 .

[6]  S. Halpern,et al.  Effect of personalization and candy incentive on response rates for a mailed survey of dermatologists. , 2012, The Journal of investigative dermatology.

[7]  G. Loosveldt,et al.  Measuring Public Opinions About Surveys , 2008 .

[8]  M. Couper,et al.  Web Surveys , 2001 .

[9]  Bernd Marcus,et al.  Compensating for Low Topic Interest and Long Surveys , 2007 .

[10]  Adam N. Joinson,et al.  Personalized salutation, power of sender and response rates to Web-based surveys , 2007, Comput. Hum. Behav..

[11]  G. Loosveldt,et al.  The effect of personalization on response rates and data quality in web surveys , 2005 .

[12]  E. Hill.  A theological perspective on social exchange theory , 1992, Journal of Religion and Health.

[13]  Robert M. Groves,et al.  Survey Errors and Survey Costs: Groves/Survey Errors , 2005 .

[14]  Yena Song,et al.  Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial , 2011, BMC medical research methodology.

[15]  Emil H Schemitsch,et al.  Internet Versus Mailed Questionnaires: A Randomized Comparison (2) , 2004, Journal of medical Internet research.

[16]  J. Feldman,et al.  Twenty-five years of health surveys: does more data mean better data? , 2007, Health affairs.

[17]  Improving response rates using a monetary incentive for patient completion of questionnaires: an observational study , 2007, BMC medical research methodology.

[18]  B. Mavis,et al.  Postal Surveys versus Electronic Mail Surveys , 1998, Evaluation & the health professions.

[19]  Terry L. Childers,et al.  Understanding mail survey response behavior: A meta-analysis. , 1991 .

[20]  Tracy L. Tuten,et al.  Prepaid and Promised Incentives in Web Surveys , 2003 .

[21]  Mick P. Couper,et al.  Technology Trends in Survey Data Collection , 2005 .

[22]  Elie A Akl,et al.  Electronic mail was not better than postal mail for surveying residents and faculty. , 2005, Journal of clinical epidemiology.

[23]  A. H. Church.  ESTIMATING THE EFFECT OF INCENTIVES ON MAIL SURVEY RESPONSE RATES: A META-ANALYSIS , 1993 .

[24]  J. Kopec,et al.  The Effect of Cash Lottery on Response Rates to an Online Health Survey among Members of the Canadian Association of Retired Persons: A Randomized Experiment , 2010, Canadian journal of public health = Revue canadienne de sante publique.

[25]  M. Galesic,et al.  Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey , 2009 .

[26]  L. Whitehead.  Methodological Issues in Internet-Mediated Research: A Randomized Comparison of Internet Versus Mailed Questionnaires , 2011, Journal of medical Internet research.

[27]  M. Clarke,et al.  Methods to increase response to postal and electronic questionnaires , 2023, The Cochrane database of systematic reviews.

[28]  Zarnie Khadjesari,et al.  Impact of Length or Relevance of Questionnaires on Attrition in Online Trials: Randomized Controlled Trial , 2011, Journal of medical Internet research.