Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial

BackgroundMinimising participant non-response in postal surveys helps to maximise the generalisability of the inferences made from the data collected. The aim of this study was to examine the effect of questionnaire length, personalisation and reminder type on postal survey response rate and quality and to compare the cost-effectiveness of the alternative survey strategies.MethodsIn a pilot study for a population study of travel behaviour, physical activity and the environment, 1000 participants sampled from the UK edited electoral register were randomly allocated using a 2 × 2 factorial design to receive one of four survey packs: a personally addressed long (24 page) questionnaire pack, a personally addressed short (15 page) questionnaire pack, a non-personally addressed long questionnaire pack or a non-personally addressed short questionnaire pack. Those who did not return a questionnaire were stratified by initial randomisation group and further randomised to receive either a full reminder pack or a reminder postcard. The effects of the survey design factors on response were examined using multivariate logistic regression.ResultsAn overall response rate of 17% was achieved. Participants who received the short version of the questionnaire were more likely to respond (OR = 1.48, 95% CI 1.06 to 2.07). In those participants who received a reminder, personalisation of the survey pack and reminder also increased the odds of response (OR = 1.44, 95% CI 1.01 to 1.95). Item non-response was relatively low, but was significantly higher in the long questionnaire than the short (9.8% vs 5.8%; p = .04). The cost per additional usable questionnaire returned of issuing the reminder packs was £23.1 compared with £11.3 for the reminder postcards.ConclusionsIn contrast to some previous studies of shorter questionnaires, this trial found that shortening a relatively lengthy questionnaire significantly increased the response. Researchers should consider the trade off between the value of additional questions and a larger sample. If low response rates are expected, personalisation may be an important strategy to apply. Sending a full reminder pack to non-respondents appears a worthwhile, albeit more costly, strategy.

[1]  D. Dillman.  Mail and internet surveys: The tailored design method, 2nd ed. , 2007 .

[2]  M. Petticrew,et al.  Perceived characteristics of the environment associated with active travel: development and testing of a new scale , 2008, The international journal of behavioral nutrition and physical activity.

[3]  B. Ainsworth,et al.  International physical activity questionnaire: 12-country reliability and validity. , 2003, Medicine and science in sports and exercise.

[4]  N. Talley,et al.  The effects of lottery incentive and length of questionnaire on health survey response rates: a randomized study. , 1999, Journal of clinical epidemiology.

[5]  S. Brophy,et al.  Interventions for latent autoimmune diabetes (LADA) in adults. , 2011, The Cochrane database of systematic reviews.

[6]  Ulf Ekelund,et al.  Estimating physical activity energy expenditure, sedentary time, and physical activity intensity by self-report in adults. , 2010, The American journal of clinical nutrition.

[7]  Edith D. de Leeuw,et al.  A comparison of nonresponse in mail, telephone, and face-to-face surveys , 1994 .

[8]  P. Chisnall.  Mail and Internet Surveys: The Tailored Design Method , 2007, Journal of Advertising Research.

[9]  E. Singer,et al.  Experiments with incentives in telephone surveys. , 2000, Public opinion quarterly.

[10]  J. Dirmaier,et al.  Incentives increased return rates but did not influence partial nonresponse or treatment outcome in a randomized trial. , 2007, Journal of clinical epidemiology.

[11]  Edith D. de Leeuw,et al.  Reducing missing data in surveys: an overview of methods , 2001 .

[12]  P. Beumont,et al.  Mode of delivery, but not questionnaire length, affected response in an epidemiological study of eating-disordered behavior. , 2004, Journal of clinical epidemiology.

[13]  F. Bull,et al.  Evaluation of the implementation of a state government community design policy aimed at increasing local walking: design issues and baseline results from RESIDE, Perth Western Australia. , 2008, Preventive medicine.

[14]  I. Ajzen.  The theory of planned behavior , 1991 .

[15]  P. Edwards,et al.  Personally addressed hand-signed letters increase questionnaire response: a meta-analysis of randomised controlled trials , 2006, BMC Health Services Research.

[16]  I. H. Harrison,et al.  A comprehensive guide , 1986 .

[17]  Michele Haynes,et al.  HABITAT: A longitudinal multilevel study of physical activity change in mid-aged adults , 2009, BMC public health.

[18]  F. Bull,et al.  An applied ecological framework for evaluating infrastructure to promote walking and cycling: the iConnect study. , 2011, American journal of public health.

[19]  Michael Sjöström,et al.  Measuring physical activity-related environmental factors: reliability and predictive validity of the European environmental questionnaire ALPHA , 2010, The international journal of behavioral nutrition and physical activity.

[20]  Francis J. Yammarino,et al.  UNDERSTANDING MAIL SURVEY RESPONSE BEHAVIOR , 2004 .

[21]  Llewellyn J. Cornelius,et al.  Designing and Conducting Health Survets. , 1998 .

[22]  S. Gates,et al.  Maximising response to postal questionnaires – A systematic review of randomised trials in health research , 2006, BMC medical research methodology.

[23]  L. Aday,et al.  Designing and conducting health surveys : a comprehensive guide , 2006 .

[24]  Terry L. Childers,et al.  Understanding mail survey response behavior: A meta-analysis. , 1991 .

[25]  M. Clarke,et al.  Methods to increase response rates to postal questionnaires. , 2007, The Cochrane database of systematic reviews.

[26]  Stanley Presser,et al.  Changes in Telephone Survey Nonresponse over the Past Quarter Century , 2005 .

[27]  Steven Cummins,et al.  EVIDENCE BASED PUBLIC HEALTH POLICY AND PRACTICE Large scale food retailing as an intervention for diet and health: quasi-experimental evaluation of a natural experiment , 2005 .

[28]  M. Petticrew,et al.  Personal and environmental correlates of active travel and physical activity in a deprived urban population , 2008, The international journal of behavioral nutrition and physical activity.