Telephone and web: mixed-mode challenge.

OBJECTIVE To explore the response rate benefits and data limitations of mixing telephone and web survey modes in a health-related research study. DATA SOURCES/STUDY SETTING We conducted a survey of salaried employees from one large employer in the manufacturing sector in the summer of 2005. STUDY DESIGN We randomized 751 subjects, all of whom had previously completed a web survey, to complete a health-related survey either by telephone (with web follow-up for nonrespondents) or over the web (with telephone follow-up). PRINCIPAL FINDINGS Survey response rates were significantly higher for the mixed mode survey than they would have been if we fielded either an exclusively telephone or web survey (25 and 12 percent higher, respectively). Telephone respondents were more likely to provide socially desirable responses to personal lifestyle questions and web respondents provided more missing data when questions were difficult or did not relate to their circumstances. Telephone respondents were also more likely to "agree" to knowledge statements and provide the same response across a series of items with similar response scales than were web respondents. CONCLUSIONS Mixed mode telephone/web surveys can substantially boost response rates over single-mode surveys. Modal differences in responses can be minimized by handling missing data options consistently in the two modes, avoiding agree/disagree formats, and not including questions on personal lifestyle or other sensitive issues.

[1]  Susan M. Rogers,et al.  Automated Self-interviewing and the Survey Measurement of Sensitive Behaviors , 1998 .

[2]  A. Bowling.  Mode of questionnaire administration can have serious effects on data quality. , 2005, Journal of public health.

[3]  Scott Fricker,et al.  An Experimental Comparison of Web and Telephone Surveys , 2005 .

[4]  D. Dillman.  Mail and internet surveys , 1999 .

[5]  Matthias Schonlau,et al.  A Comparison Between Responses From a Propensity-Weighted Web Survey and an Identical RDD Survey , 2004 .

[6]  J. Newhouse,et al.  How Sophisticated Are Consumers About the Medical Care Delivery System? , 1981, Medical care.

[7]  T. Olmstead,et al.  The impact of managed care on substance abuse treatment services. , 2004, Health services research.

[8]  Roger Tourangeau,et al.  Survey research and societal change. , 2004, Annual review of psychology.

[9]  Michael W. Link,et al.  Alternative Modes for Health Surveillance Surveys: An Experiment with Web, Mail, and Telephone , 2005, Epidemiology.

[10]  Judith H Hibbard,et al.  Development and testing of a short form of the patient activation measure. , 2005, Health services research.

[11]  J. Hibbard,et al.  Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. , 2004, Health services research.

[12]  Michael W. Link,et al.  Call Screening: Is It Really a Problem for Survey Research? , 1999 .

[13]  Humphrey Taylor,et al.  Does Internet Research Work? , 2000 .

[14]  Tommy L. Brown,et al.  Effects of Category Order on Answers in Mail and Telephone Surveys1 , 2010 .

[15]  Don A. Dillman,et al.  Survey Mode as a Source of Instability in Responses across Surveys , 2005 .

[16]  B. Rimer,et al.  Methods and Strategies to Recruit African Americans into Cancer Prevention Surveillance Studies , 2005, Cancer Epidemiology Biomarkers & Prevention.

[17]  C. McHorney,et al.  Comparisons of the Costs and Quality of Norms for the SF-36 Health Survey Collected by Mail Versus Telephone Interview: Results From a National Survey , 1994, Medical care.

[18]  Matthias Schonlau,et al.  Advantages and Disadvantages of Internet Research Surveys: Evidence from the Literature , 2002 .

[19]  S. M. Rogers,et al.  Reducing bias in telephone survey estimates of the prevalence of drug use: a randomized trial of telephone audio-CASI. , 2005, Addiction.

[20]  M. Couper.  A REVIEW OF ISSUES AND APPROACHES , 2000 .

[21]  C. Steeh,et al.  Are They Really as Bad as They Seem? Nonresponse Rates at the End of the Twentieth Century , 2001 .

[22]  J. Krosnick,et al.  National Surveys Via Rdd Telephone Interviewing Versus the Internet Comparing Sample Representativeness and Response Quality , 2009 .

[23]  J. Hibbard,et al.  Which consumers are ready for consumer-directed health plans? , 2006 .

[24]  E. Leeuw,et al.  To mix or not to mix data collection modes in surveys. , 2005 .

[25]  Don A. Dillman,et al.  Understanding differences in people's answers to telephone and mail surveys , 1996 .

[26]  J. R. Hochstim.  A Critical Comparison of Three Strategies of Collecting Data from Households , 1967 .

[27]  Michael W. Link,et al.  Effects of survey mode on self-reports of adult alcohol consumption: a comparison of mail, web and telephone approaches. , 2005, Journal of studies on alcohol.