Survey Mode as a Source of Instability in Responses across Surveys

Changes in survey mode for conducting panel surveys may contribute significantly to survey error. This article explores the causes and consequences of such changes in survey mode. The authors describe how and why the choice of survey mode often causes changes to be made to the wording of questions, as well as the reasons that identically worded questions often produce different answers when administered through different modes. The authors provide evidence that answers may changeas a result of different visual layouts for otherwise identical questions and suggest ways to keep measurement the same despite changes in survey mode.

[1]  P. Chisnall.  Mail and Internet Surveys: The Tailored Design Method , 2007, Journal of Advertising Research.

[2]  D. Dillman.  Mail and internet surveys: The tailored design method, 2nd ed. , 2007 .

[3]  Don A. Dillman,et al.  The Influence of Graphical and Symbolic Language Manipulations on Responses to Self-Administered Questions , 2004 .

[4]  Cleo D. Redline,et al.  Improving Navigational Performance in U.S. Census 2000 by Altering the Visually Administered Languages of Branching Instructions , 2003 .

[5]  Jolene D. Smyth,et al.  Multiple Answer Questions in Self-Administered Surveys: The Use of Check-All-That-Apply and Forced-Choice Question Formats * , 2003 .

[6]  Leah Melani Christian,et al.  The influence of visual layout on scalar questions in web surveys , 2003 .

[7]  Don A. Dillman,et al.  Presidential Address: Navigating the Rapids of Change: Some Observations on Survey Methodology in the Early Twenty-First Century , 2002 .

[8]  Sherrill A. Norris,et al.  CUSTOMER SATISFACTION WITH INTERNET AND IVR AS CENSUS DATA COLLECTION TOOLS , 2002 .

[9]  Roger Tourangeau,et al.  Self-Administered Questions by Telephone: Evaluating Interactive Voice Response , 2001 .

[10]  Cleo D. Redline,et al.  The Influence of Alternative Visual Designs on Respondents ’ Performance with Branching Instructions in Self-Administered Questionnaires , 2000 .

[11]  C. Shettle,et al.  Monetary Incentives in U. S. Government Surveys , 1999 .

[12]  F. J. Fowler,et al.  Mode Effects in a Survey of Medicare Prostate Surgery Patients , 1998 .

[13]  Don A. Dillman,et al.  Understanding differences in people's answers to telephone and mail surveys , 1996 .

[14]  Norman M. Bradburn,et al.  DO RESPONDENTS REALLY “MARK ALL THAT APPLY” ON SELF-ADMINISTERED QUESTIONS? , 1994 .

[15]  D. Leeuw,et al.  Data Quality in Mail, Telephone and Face to Face Surveys. , 1992 .

[16]  Norbert Schwarz,et al.  Assimilation and Contrast Effects in Part-Whole Question Sequences: A Conversational Logic Analysis , 1991 .

[17]  R. Groves,et al.  Telephone Survey Methodology. , 1990 .

[18]  J. Krosnick,et al.  AN EVALUATION OF A COGNITIVE THEORY OF RESPONSE-ORDER EFFECTS IN SURVEY MEASUREMENT , 1987 .

[19]  F. Strack,et al.  A comparison of response effects in self-administered and telephone surveys , 1987 .

[20]  S. Presser,et al.  Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context , 1996 .

[21]  P. Miller.  Alternative Question Forms for Attitude Scale Questions in Telephone Interviews , 1984 .

[22]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .

[23]  J. R. Hochstim.  A Critical Comparison of Three Strategies of Collecting Data from Households , 1967 .