Face-to-Face Interviews and Mailed Questionnaires: The Net Difference in Response Rate

Traditionally, as reflected in methodology textbook reviews of the survey nonresponse literature, interviews have been thought to achieve substantially higher response rates than mailed questionnaires. The paper describes a correlational design for assessing the typical response difference between these two forms of data collection, after controls for other factors known to affect survey response. Attention is given to change over time in the response difference between interviews and questionnaires. In the primary data set, the net response due to data collection amounts to some 7.5 percentage points, in the middle (30-70) range of survey response. The analysis quantifies the decline over time in interview response, and shows mailed questionnaires to be free from decline. A second data set reproduces the essential findings. John Goyder is Associate Professor, Department of Sociology, University of Waterloo, Ontario, Canada N2L 3G1. The work reported herein was supported by a Canada Council Leave Fellowship held by the author during a stay at Cambridge University in 1978, and by a UW-SSHRC grant awarded in 1981. The author is greatly indebted to Cathie Marsh and others at the Social and Political Sciences Committee, Cambridge, for hospitality and research resources. Research Assistants were Norleen Heyzer and Kerry Davies at Cambridge and Jean Leiper and Olorunfe Taylor-Cole at Waterloo. Public Opinion Quarterly Vol. 49:234-252 ?) by the Trustees of Columbia University Published by Elsevier Science Publishing Co., Inc 0033-362X/85/0049-234/$2.50 This content downloaded from 157.55.39.181 on Thu, 29 Sep 2016 05:51:04 UTC All use subject to http://about.jstor.org/terms INTERVIEWS AND MAILED QUESTIONNAIRES 235 (1981:150), the 50 percent ceiling holds true even allowing for follow-ups. Others, however, set the upper limit markedly higher. Fitzgerald and Cox (1975:53), for example, choose 60 percent, while, for True (1983: 218), "estimates of likely return rates range to 70 percent," a maximum also reported in Goode and Hatt (1952:173) and Cole (1980:62). Sanders and Pinhey (1983:152-53) cite 60 percent as an average and note that "some researchers have reported returns of 80 percent and more." Consistent 70 percent returns from general populations are not unrealistic, argue Weisberg and Bowen (1977:58), given energetic follow-ups (also Miller, 1977:79-81). Estimated typical completion rate on interview surveys is 70-80 percent for Kidder (1981:150), 80-85 percent for Babbie (1973:171), 80-90 percent for Orenstein and Phillips (1978:229), over 90 percent according to Parten (1950:351), "about 95 percent" for Nachmias and Nachmias (1976:107), and "as much as 85 percent" for Sanders and Pinhey (1983: 154). Weisberg and Bowen (1977:36), introducing a factor to be emphasized herein, quote 90 percent interview response as "typical" during the 1950s, with rates "in the 80 percent range" common today. The greater response due to data collection by interview rather than by questionnaire could, from the above estimates, amount to anywhere from zero to some 60 percentage points. In noting the variability of the textbook estimates, we do not deny that the accumulated wisdom on survey methodology presents a broad consensus that interview response is generally greater than mailed questionnaire returns, nor that each of the typical response figures cited by different authors accurately reflects at least some aspect of reality. The review of textbooks is effective, however, in emphasizing the variability in survey response according to both form of data collection and other factors. A "typical" survey response rate exists only insofar as there is a typical salience level of topic, sponsor, target population, and set of field procedures. In the unsystematic comparisons of questionnaire and interview response presented in methodology textbooks, these various factors have varied haphazardly. The argument in this paper is that the survey methodology literature is now in a position to establish a net effect for form of data collection, controlling for other determinants of survey response by means of multivariate analysis. We develop this methodology herein, and establish the net margin between interviews and questionnaires, both as averaged over the history of survey research and as mediated over time.

[1]  Duane F. Alwin,et al.  Making Errors in Surveys , 1977 .

[2]  Alan M. Sear,et al.  Questionnaire Response Rate: A Methodological Analysis , 1969 .

[3]  Robert M. Groves,et al.  Actors and Questions in Telephone and Personal Interview Surveys , 1979 .

[4]  W. J. Goode,et al.  Methods in Social Research , 1952 .

[5]  M. J. O'neil.  Estimating the Nonresponse BiasDue to Refusals in Telephone Surveys , 1979 .

[6]  J. Lehman.  Report on the ASA Conference on Surveys of Human Populations , 1974 .

[7]  F. Kviz.  Toward a Standard Definition of Response Rate , 1977 .

[8]  E. Singer.  Informed consent: consequences for response rate and response quality in social surveys. , 1978, American sociological review.

[9]  H. Gaudet,et al.  Who escapes the personal investigator , 1940 .

[10]  C. Steeh.  Trends in Nonresponse Rates, 1952–1979 , 1981 .

[11]  Thomas W. Mangione,et al.  Collecting Sensitive Data , 1982 .

[12]  W. Fairweather,et al.  Physician response rates to mail and personal interview surveys. , 1979, Public opinion quarterly.

[13]  Delbert Miller,et al.  Handbook of research design and social measurement , 1993 .

[14]  G. Gordon,et al.  Maximizing Returns on Mail Questionnaires , 1958 .

[15]  K. Michael Cummings,et al.  Random Digit Dialing: A Sampling Technique for Telephone Surveys , 1979 .

[16]  W. Kephart,et al.  Increasing the Responses to Mail Questionnaires: A Research Study , 1958 .

[17]  R. F. Sletto.  Pretesting of Questionnaires , 1940 .

[18]  G. Sjoberg.  A Questionnaire on Questionnaires , 1954 .

[19]  Seymour Sudman,et al.  Response Effects in Surveys: A Review and Synthesis , 1974 .

[20]  Pauline V. Young.  Scientific social surveys and research : an introduction to the background, content, methods, and analysis of social studies / Pauline V. Young, Calvin F. Schmidt , 2012 .

[21]  Seymour Sudman,et al.  Improving Interview Method and Questionnaire Design. , 1982 .

[22]  Chava Nachmias,et al.  Research Methods in the Social Sciences , 1976 .

[23]  Klaus Eichner,et al.  Predicting Reponse Rates to Mailed Questionnaires , 1981 .

[24]  Earl R. Babbie,et al.  Survey Research Methods , 1984 .

[25]  Paul B. Sheatsley,et al.  Nonresponse Bias for Attitude Questions , 1981 .

[26]  S. Labovitz,et al.  Introduction to social research , 1971 .

[27]  K. Bailey.  Methods of Social Research , 1978 .

[28]  June Audrey True,et al.  Finding Out: Conducting and Evaluating Social Research , 1983 .

[29]  D. Dillman,et al.  Increasing Mail Questionnaire Response: A Four State Comparison , 1974 .

[30]  T. DeMaio.  Refusals: Who, Where and Why , 1980 .

[31]  T. Heberlein,et al.  Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. , 1978 .

[32]  A. S. Linsky.  STIMULATING RESPONSES TO MAILED QUESTIONNAIRES: A REVIEW , 1975 .

[33]  William B. Sanders,et al.  The conduct of social research , 1979 .

[34]  J. Goyder,et al.  Further Evidence on Factors Affecting Response Rates to Mailed Questionnaires , 1982 .

[35]  C. Scott.  Research on Mail Surveys , 1961 .

[36]  G. Vigderhous.  Analysis of Patterns of Response To Mailed Questionnaires , 1977 .

[37]  H. Ross.  THE INACCESSIBLE RESPONDENT: A NOTE ON PRIVACY IN CITY AND COUNTRY , 1963 .

[38]  Robert M. Groves,et al.  Surveys by Telephone: A National Comparison With Personal Interviews. , 1980 .

[39]  M. Parten,et al.  Surveys, Polls, and Samples: Practical Procedures. , 1950 .

[40]  T. Heberlein,et al.  Is a Questionnaire Necessary in a Second Mailing , 1981 .

[41]  Don A. Dillman,et al.  Reducing Refusal Rates for Telephone Interviews , 1976 .

[42]  Carl F. Reuss,et al.  Differences Between Persons Responding and Not Responding to a Mailed Questionnaire , 1943 .

[43]  Wayne W. Daniel,et al.  Nonresponse in Sociological Surveys , 1975 .

[44]  S. Fiske,et al.  The Handbook of Social Psychology , 1935 .

[45]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[46]  S. Cole.  The sociological method: An introduction to the science of sociology , 1980 .

[47]  Jon A. Krosnick,et al.  An introduction to survey research and data analysis, 2nd ed. , 1977 .

[48]  D. Champion,et al.  Methods and issues in social research , 1976 .

[49]  A. Orenstein,et al.  Understanding Social Research: An Introduction , 1978 .

[50]  Kenneth R. Pfeiffer,et al.  Introductory Statistics for the Behavioral Sciences.@@@Basic Statistics for the Behavioral Sciences.@@@Selltiz, Wrightsman and Cook's Research Methods in Social Relations. , 1982 .

[51]  L. Kanuk,et al.  Mail Surveys and Response Rates: A Literature Review , 1975 .

[52]  L. Fuller,et al.  I Hear You Knocking But You Can't Come In , 1982 .

[53]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .