A Quantitative Review of Research Design Effects on Response Rates to Questionnaires

A comprehensive literature review of techniques used to increase response rates to questionnaires was conducted. Conclusions were based on arithmetic combination of 497 response rates found in 93 journal articles. Response rates were found to be increased by personal and telephone (versus mail) surveys, either prepaid or promised incentives, nonmonetary premiums and rewards, and increasing amounts of monetary reward. Other facilitators that increased responding were preliminary notification, foot-in-the-door techniques, personalization, and followup letters. Several facilitators were found to be ineffective. An estimated magnitude of effect was calculated for each design variation.

[1]  Irene Hess,et al.  A “Replacement” Procedure for Reducing the Bias of Nonresponse , 1959 .

[2]  William F. O'Dell.  Personal Interviews or Mail Panels? , 1962 .

[3]  Charles F. Cannell,et al.  COMPARISON OF A SELF-ENUMERATIVE PROCEDURE AND A PERSONAL INTERVIEW: A VALIDITY STUDY , 1963 .

[4]  Morton L. Brown.  USE OF A POSTCARD QUERY IN MAIL SURVEYS , 1965 .

[5]  E. McDonagh,et al.  A COMPARISON OF MAILED QUESTIONNAIRES AND SUBSEQUENT STRUCTURED INTERVIEWS , 1965 .

[6]  J. Freedman,et al.  Compliance without pressure: the foot-in-the-door technique. , 1966, Journal of personality and social psychology.

[7]  James E. Stafford.  Influence of Preliminary Contact on Mail Returns , 1966 .

[8]  Maryline Meyer,et al.  TIMING POSTCARD FOLLOW-UPS IN MAIL-QUESTIONNAIRE SURVEYS , 1966 .

[9]  Thomas R. Wotruba,et al.  Monetary Inducements and Mail Questionnaire Response , 1966 .

[10]  J. R. Hochstim.  A Critical Comparison of Three Strategies of Collecting Data from Households , 1967 .

[11]  Neil M. Ford.  The Advance Letter in Mail Surveys , 1967 .

[12]  Harry M. Scoble,et al.  MAIL QUESTIONNAIRE EFFICIENCY: CONTROLLED REDUCTION OF NONRESPONSE , 1967 .

[13]  Alan M. Sear,et al.  Questionnaire Response Rate: A Methodological Analysis , 1969 .

[14]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[15]  J. Kirscht,et al.  INTERVIEWING A NATIONAL SAMPLE BY LONG-DISTANCE TELEPHONE , 1969 .

[16]  Barbara Snell Dohrenwend.  AN EXPERIMENTAL STUDY OF PAYMENTS TO RESPONDENTS , 1970 .

[17]  Alan R. Andreasen.  PERSONALIZING MAIL QUESTIONNAIRE CORRESPONDENCE , 1970 .

[18]  F. W. King.  Anonymous versus identifiable questionnaires in drug usage surveys. , 1970, The American psychologist.

[19]  J. B. Kernan.  ARE “BULK-RATE OCCUPANTS” REALLY UNRESPONSIVE? , 1971 .

[20]  L. M. Aleamoni,et al.  Effect of Personal Signature on the Initial Rate of Return of a Mailed Questionnaire. , 1971 .

[21]  David J. Pucel,et al.  Questionnaire Follow Up Returns as a Function of Incentives and Responder Characteristics. , 1971 .

[22]  C. Hendrick,et al.  Effectiveness of ingratiation tactics in a cover letter on mail questionnaire response , 1972 .

[23]  Don A. Dillman,et al.  INCREASING MAIL QUESTIONNAIRE RESPONSE IN LARGE SAMPLES OF THE GENERAL PUBLIC , 1972 .

[24]  R. Parsons,et al.  THE EFFECT OF ADVANCE NOTICE IN MAIL SURVEYS OF HOMOGENEOUS GROUPS , 1972 .

[25]  Frederick Wiseman,et al.  METHODOLOGICAL BIAS IN PUBLIC OPINION SURVEYS , 1972 .

[26]  D. R. Berdie.  Questionnaire length and response rate. , 1973 .

[27]  D. Cahalan,et al.  BY-MAIL VS. FIELD SELF-ADMINISTERED QUESTIONNAIRES: AN ARMED FORCES SURVEY , 1973 .

[28]  Frederick Wiseman,et al.  Factor Interaction Effects in Mail Survey Response Rates , 1973 .

[29]  W. S. Blumenfeld.  Effect of Appearance of Correspondence on Response Rate to a Mail Questionnaire Survey , 1973 .

[30]  J. Hackler,et al.  DOLLARS, DISSONANCE, AND SURVEY RETURNS , 1973 .

[31]  E. H. Carpenter.  Personalizing Mail SurveysA Replication and Reassessment , 1974 .

[32]  R. Ferber,et al.  A Comparison of Alternative Procedures for Collecting Consumer Expenditure Data for Frequently Purchased Products , 1974 .

[33]  John F. Veiga,et al.  Getting the mail questionnaire returned: Some practical research considerations. , 1974 .

[34]  Don A. Dillman,et al.  Contribution of personalization to mail questionnaire response as an element of a previously tested method. , 1974 .

[35]  S. Huck,et al.  Using monetary inducements to increase response rates from mailed surveys: A replication and extension of previous research. , 1974 .

[36]  D. Dillman,et al.  Increasing Mail Questionnaire Response: A Four State Comparison , 1974 .

[37]  Bruce J. Walker,et al.  Effects of alternative follow-up procedures on mail survey response rates. , 1974 .

[38]  P. Pliner,et al.  Compliance without pressure: Some further data on the foot-in-the-door technique , 1974 .

[39]  Mathew Hauck,et al.  Locating a Sample by Random Digit Dialing , 1974 .

[40]  Carol H. Fuller,et al.  Effect of anonymity on return rate and response bias in a mail survey. , 1974 .

[41]  Robert V. Stover,et al.  Hand Delivery of Self-Administered Questionnaires , 1974 .

[42]  Eli P. Cox,et al.  Reappraising Mail Survey Response Rates , 1974 .

[43]  Jagdish N. Sheth,et al.  Follow-up Methods, Questionnaire Length, and Market Differences in Mail Surveys , 1975 .

[44]  M. Cunningham,et al.  To comply or not comply: testing the self-perception explanation of the "foot-in-the-door" phenomenon. , 1975, Journal of personality and social psychology.

[45]  B. Gelb.  Incentives to Increase Survey Returns: Social Class Considerations , 1975 .

[46]  S. Sherman,et al.  Effects of initial request size and timing of a second request on compliance: the foot in the door and the door in the face. , 1975, Journal of personality and social psychology.

[47]  J. Sheth,et al.  Erratum: Impact of Questionnaire Length Follow-Up Methods, and Geographical Location on Response Rate to a Mail Survey. , 1975 .

[48]  Michael J. Houston,et al.  The Negative Effects of Personalization on Response Patterns in Mail Surveys , 1975 .

[49]  J. F. Anderson,et al.  Effects on response rates of formal and informal questionnaire follow-up techniques. , 1975 .

[50]  A. S. Linsky.  STIMULATING RESPONSES TO MAILED QUESTIONNAIRES: A REVIEW , 1975 .

[51]  J. Armstrong.  Monetary Incentives in Mail Surveys , 1975 .

[52]  Robert A. Peterson,et al.  An experimental investigation of mail-survey responses , 1975 .

[53]  L. Kanuk,et al.  Mail Surveys and Response Rates: A Literature Review , 1975 .

[54]  C. G. Walters,et al.  The Effect On Return Rate of Messages Explaining Monetary Incentives in Mail Questionnaire Studies , 1976 .

[55]  Theresa F. Rogers,et al.  Interviews by Telephone and in Person Quality of Responses and Field Performance , 1976 .

[56]  Jr. James R. Henley.  Response Rate to Mail Questionnaires with a Return Deadline , 1976 .

[57]  William J. Whitmore.  Mail Survey Premiums and Response Bias , 1976 .

[58]  C. Schewe,et al.  Prepaid vs. Promised Monetary Incentives to Questionnaire Response Further Evidence , 1976 .

[59]  Clive Seligman,et al.  Relationship between compliance in the foot-in-the-door paradigm and size of first request. , 1976 .

[60]  N. Bradburn,et al.  An Investigation of Interview Method, Threat and Response Distortion , 1976 .

[61]  J. House,et al.  Increasing Mail Questionnaire Response: A Controlled Replication and Extension , 1977 .

[62]  Jerome B. Kernan,et al.  Compliance with an Interview Request: A Foot-in-the-Door, Self-Perception Interpretation , 1977 .

[63]  J. E. Swan,et al.  Anonymity and Response by Salespeople to a Mail Questionnaire , 1977 .

[64]  T. Walsh.  Selected Results from the 1972–73 Diary Surveys , 1977 .

[65]  Michael S. Goodstadt,et al.  Mail Survey Response Rates: Their Manipulation and Impact , 1977 .

[66]  T. Vocino,et al.  Three Variables in Stimulating Responses to Mailed Questionnaires , 1977 .

[67]  R. Wildman,et al.  Effects of Anonymity and Social Setting on Survey Responses , 1977 .

[68]  John R. Nevin,et al.  The Effects of Source and Appeal on Mail Survey Response Patterns , 1977 .

[69]  Carol A. Scott.  Modifying Socially-Conscious Behavior: The Foot-in-the-Door Technique , 1977 .

[70]  M. A. Jolson.  How to Double or Triple Mail-Survey Response Rates , 1977 .

[71]  Richard K. Burdick,et al.  Advance Correspondence and Error in Mail Surveys , 1977 .

[72]  M. A. Mcginnis,et al.  Mail Survey Response Rate and Bias: The Effect of Home versus Work Address , 1977 .

[73]  W. Tullar,et al.  A Factor Interactive Investigation of Mail Survey Response Rates from a Commercial Population , 1977 .

[74]  Dan H. Robertson,et al.  A New Method of Increasing Mail Survey Responses: Contributions to Charity , 1978 .

[75]  E. Singer.  Informed consent: consequences for response rate and response quality in social surveys. , 1978, American sociological review.

[76]  David Labrecque,et al.  A Response Rate Experiment Using Mail Questionnaires , 1978 .

[77]  Bill Williams.  A Sampler on Sampling , 1978 .

[78]  D. Horvitz,et al.  The Use of Monetary Incentives in National Assessment Household Surveys , 1978 .

[79]  R. Forthofer,et al.  Further Evidence on Using a Deadline to Stimulate Responses to a Mail Survey; Readership and Coverage of Science and Technology in Newspapers; The Uses and Gratifications Approach to Mass Communications Research , 1979 .

[80]  R. Rosenthal.  Combining results of independent studies. , 1978 .

[81]  M. Pressley,et al.  Care needed when selecting response inducements in mail surveys of commercial populations , 1978 .

[82]  Lewis Mandell,et al.  Some Insight into the Underreporting of Financial Data by Sample Survey Respondents , 1978 .

[83]  Harris Cooper,et al.  Statistically Combining Independent Studies: A Meta-Analysis of Sex Differences in Conformity Research , 1979 .

[84]  O. C. Ferrell,et al.  Response Rates and Perceived Questionnaire Length in Mail Surveys , 1979 .

[85]  W. Fairweather,et al.  Physician response rates to mail and personal interview surveys. , 1979, Public opinion quarterly.

[86]  Hershey H. Friedman,et al.  The effects of a monetary incentive and the ethnicity of the sponsor's signature on the rate and quality of response to A mail survey , 1979 .

[87]  R. Kenneth Godwin,et al.  The Consequences ofLarge Monetary Incentivesin Mail Surveys of Elites , 1979 .

[88]  O. C. Ferrell,et al.  A Reassessment of the Effects of Appeals on Response to Mail Surveys , 1980 .

[89]  C. P. Rao,et al.  The Effect of Monetary Inducement on Mailed Questionnaire Response Quality , 1980 .

[90]  J. R. Lang,et al.  Sample Composition Bias and Response Bias in a Mail Survey: A Comparison of Inducement Methods , 1980 .

[91]  Robert A. Hansen,et al.  A Self-Perception Interpretation of the Effect of Monetary and Nonmonetary Incentives on Mail Survey Respondent Behavior , 1980 .

[92]  Jacob Hornik,et al.  Time Cue and Time Perception Effect on Response to Mail Surveys , 1981 .

[93]  Kevin F. McCrohan,et al.  A Cost/benefit Approach to Postage used on Mail Questionnaires , 1981 .

[94]  David L. Rados,et al.  Effects of Foot-in-the-Door, Cash Incentives, and Followups on Survey Response , 1981 .

[95]  T. Heberlein,et al.  Is a Questionnaire Necessary in a Second Mailing , 1981 .

[96]  Harris Cooper,et al.  Scientific Guidelines for Conducting Integrative Research Reviews , 1982 .