Mixed mode designs: finding the balance between nonresponse bias and mode effects

Mixed mode designs in survey research can be used to lower nonresponse rates and prevent nonresponse bias. The main disadvantage is that the use of different modes of data collection can bias the results. By making a distinction between selection processes and transformation processes, it is possible to model the effects of a mixed mode design on the quality of survey data. Using data gathered on the occasion of the 1998 Dutch national elections, we show that mixed mode surveys and adding the possibility for nonrespondents to answer so called central questions, leads to higher response rates and to less nonresponse bias. Although we do find a higher level of response bias for the face-to-face mode as compared to the telephone and mail modes, the inclusion of the respondents that answered the face-to-face interview still leads to better estimates of the true population values than when these people are left out.

[1]  J. Conley.  Asking questions: A practical guide to questionnaire design , 1983 .

[2]  L. Midanik,et al.  Effects of telephone versus face-to-face interview modes on reports of alcohol consumption. , 2000, Addiction.

[3]  F. Strack,et al.  The impact of administration mode on response effects in survey measurement , 1991 .

[4]  J. R. Hochstim.  A Critical Comparison of Three Strategies of Collecting Data from Households , 1967 .

[5]  Paul Biemer,et al.  Nonresponse Bias and Measurement Bias in a Comparison of Face to Face and Telephone Interviewing , 2001 .

[6]  Jeroen K. Vermunt,et al.  LEM: A general program for the analysis of categorical data. Users manual , 1997 .

[7]  Norman M. Bradburn,et al.  DO RESPONDENTS REALLY “MARK ALL THAT APPLY” ON SELF-ADMINISTERED QUESTIONS? , 1994 .

[8]  P. Abramson,et al.  Race-Related Differences in Self-Reported and Validated Turnout in 1986 , 1989, The Journal of Politics.

[9]  Robert M. Groves,et al.  Surveys by Telephone: A National Comparison with Personal Interviews , 1980 .

[10]  Wendy Sykes,et al.  The problems of non-coverage and unlisted numbers in telephone surveys in Britain , 1987 .

[11]  T. Heberlein,et al.  Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. , 1978 .

[12]  M. Crask,et al.  MAIL SURVEY RESPONSE RATE A META-ANALYSIS OF SELECTED TECHNIQUES FOR INDUCING RESPONSE , 1988 .

[13]  Robert F. Belli,et al.  Reducing vote overreporting in surveys : Social desirability, memory failure, and source monitoring , 1999 .

[14]  Sören Holmberg,et al.  Self-Reported Turnout and Voter Validation , 1991 .

[15]  R. Montjoy,et al.  Overreporting voting: why it happens and why it matters. , 2001, Public opinion quarterly.

[16]  J. Hagenaars.  Loglinear Models with Latent Variables , 1993 .

[17]  Seymour Sudman,et al.  Response Effects in Surveys: A Review and Synthesis , 1974 .

[18]  F. J. Fowler,et al.  Mode Effects in a Survey of Medicare Prostate Surgery Patients , 1998 .

[19]  On the validity and reliability of self-reported vote: validity without reliability? , 1997 .

[20]  S. Presser.  CAN CHANGES IN CONTEXT REDUCE VOTE OVERPORTING IN SURVEYS , 1990 .

[21]  J. R. Hochstim,et al.  PERSONAL FOLLOW-UP IN A MAIL SURVEY: ITS CONTRIBUTION AND ITS COST , 1970 .

[22]  J. Soriano,et al.  Effect of the method of administration, mail or telephone, on the validity and reliability of a respiratory health questionnaire. The Spanish Centers of the European Asthma Study. , 1998, Journal of clinical epidemiology.

[23]  N. Day,et al.  Maximizing Response to Surveys in Health Program Evaluation At Minimum Cost Using Multiple Methods , 1995 .

[24]  Edith D. de Leeuw,et al.  A comparison of nonresponse in mail, telephone, and face-to-face surveys , 1994 .

[25]  P. Abramson,et al.  Race-Related Differences in Self-Reported and Validated Turnout , 1984, The Journal of Politics.

[26]  Matthew N. Beckmann,et al.  What Leads to Voting Overreports? Contrasts of Overreporters to Validated Voters and Admitted Nonvoters in the American National Election Studies , 2001 .

[27]  Larry A. Kroutil,et al.  Mode Effects in Self-Reported Mental Health Data , 2001 .

[28]  D. Dillman,et al.  Mail and telephone surveys , 1978 .

[29]  L. Rips,et al.  The Psychology of Survey Response , 2000 .

[30]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[31]  William T. Harbaugh.  If people vote because they like to, then why do so many of them lie? , 1996 .

[32]  J. Goyder,et al.  Face-to-Face Interviews and Mailed Questionnaires: The Net Difference in Response Rate , 1985 .

[33]  Paul P. Biemer,et al.  Introduction to Survey Quality , 2003 .

[34]  C. Shettle,et al.  Monetary Incentives in U. S. Government Surveys , 1999 .

[35]  S. Presser,et al.  Data collection mode and social desirability bias in self-reported religious attendance : Church attendance in the United States , 1998 .

[36]  Seymour Sudman,et al.  Autobiographical memory and the validity of retrospective reports , 1995 .

[37]  EDITH D. DE LEEUW,et al.  The Influence of Data Collection Method on Structural Models , 1996 .