Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates? An Experimental Examination

Abstract Survey research has long grappled with the concept of survey mode preference: the idea that a re-spondent may prefer to participate in one survey mode over another. This article experimentally examines the effect of mode preference on response, contact, and cooperation rates; mode choice; and data collection efficiency. Respondents to a 2008 telephone survey ( n = 1,811; AAPOR RR3 = 38 percent) were asked their mode preference for future survey participation. These respondents were subsequently followed up in 2009 with two independent survey requests. The first follow-up survey request was another telephone survey ( n = 548; AAPOR RR2 = 55.5 percent). In the second follow-up survey ( n = 565; AAPOR RR2 = 46.0 percent), respondents were randomly assigned to one of four mode treatments: Web only, mail only, Web followed by mail, and mail followed by Web. We find that mode preference predicts participation in Web and phone modes, cooperation in phone mode (where contact and cooperation can be disentangled), and the selection of a mode when given the option of two modes. We find weak and mixed evidence about the relationship be-tween mode preference and reduction of field effort. We discuss the important implications these findings have for mixed mode surveys.

[1]  Robert M. Groves,et al.  An examination of within-person variation in response propensity over the data collection field period , 2012 .

[2]  Robert M. Rosenberg,et al.  American Association for Public Opinion Research , 2012 .

[3]  Morgan M. Millar,et al.  Improving Response To Web and Mixed-Mode Surveys , 2011 .

[4]  Vasja Vehovar,et al.  Nonresponse in Web Surveys , 2011, International Encyclopedia of Statistical Science.

[5]  Angela P. Wetzel.  Internet, mail, and mixed‐mode surveys: The tailored design method , 2010 .

[6]  Robert D. Tortora,et al.  Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet , 2009 .

[7]  Michael W. Link,et al.  A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys , 2008 .

[8]  P. Lavrakas.  Surveys by Telephone , 2008 .

[9]  S. Gesell,et al.  Test of a Web and Paper Employee Satisfaction Survey : Comparison of Respondents and Non-Respondents , 2008 .

[10]  Samuel Garrett-Jones,et al.  How Demographic Characteristics Affect Mode Preference in a Postal/Web Mixed-Mode Survey of Australian Researchers , 2007 .

[11]  R. Groves.  Nonresponse Rates and Nonresponse Bias in Household Surveys , 2006 .

[12]  E. Leeuw,et al.  To mix or not to mix data collection modes in surveys. , 2005 .

[13]  Michael D. Kaplowitz,et al.  A Comparison of Web and Mail Survey Response Rates , 2004 .

[14]  Robyn Attewell,et al.  A comparison of an electronic version of the SF-36 General Health Questionnaire to the standard paper version , 2002, Quality of Life Research.

[15]  Thomas I. Miller,et al.  Citizen Surveys on the Web , 2002 .

[16]  K. Stein,et al.  AN EXPERIMENT IN DATA COLLECTION MODE PREFERENCE OF LONG-TERM CANCER SURVIVORS , 2002 .

[17]  John Van Hoewyk,et al.  A multivariate technique for multiply imputing missing values using a sequence of regression models , 2001 .

[18]  E. Singer,et al.  Leverage-saliency theory of survey participation: description and an illustration. , 2000, Public opinion quarterly.

[19]  E. Singer,et al.  The effects of response rate changes on the index of consumer sentiment. , 2000, Public opinion quarterly.

[20]  Paul C. Hardin,et al.  Paper presented at the Annual Meeting of the American , 1999 .

[21]  Don A. Dillman,et al.  Development of a Standard E-Mail Methodology: Results of an Experiment , 1998 .

[22]  Don A. Dillman,et al.  INFLUENCE OF AN INVITATION TO ANSWER BY TELEPHONE ON RESPONSE TO CENSUS QUESTIONNAIRES , 1994 .