ESTIMATING THE EFFECT OF INCENTIVES ON MAIL SURVEY RESPONSE RATES: A META-ANALYSIS

This article reports the results of a meta-analysis of 38 experimental and quasi-experimental studies that implemented some form of mail survey incentive in order to increase response rates. A total of 74 observations or cases were classified into one of four types of incentive groups: those using prepaid monetary or nonmonetary rewards included with the initial survey mailing and those using monetary or nonmonetary rewards as conditional upon the return of the survey. Results were generated using an analysis of variance approach. The overall effect size across the 74 observations was reported as low to moderate at d = .241. When compared across incentive types, only those surveys that included rewards (both monetary and nonmonetary) in the initial mailing yielded statistically significant estimates of effect size (d = .347, d = .136). The average increase in response rates over control conditions for these types of incentives was 19.1 percent and 7.9 percent, respectively. There was no evidence of any impact for those incentive types offering rewards contingent upon the return of the survey.

[1]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[2]  W J Gunn,et al.  Physician response rates to a telephone survey: effects of monetary incentive level. , 1981, Public opinion quarterly.

[3]  M. Crask,et al.  MAIL SURVEY RESPONSE RATE A META-ANALYSIS OF SELECTED TECHNIQUES FOR INDUCING RESPONSE , 1988 .

[4]  R. Ferber,et al.  A Comparison of Alternative Procedures for Collecting Consumer Expenditure Data for Frequently Purchased Products , 1974 .

[5]  J. Knox.  MAXIMIZING RESPONSES TO MAIL QUESTIONNAIRES: A NEW TECHNIQUE , 1951 .

[6]  B. Gelb.  Incentives to Increase Survey Returns: Social Class Considerations , 1975 .

[7]  H. Cooper,et al.  A Quantitative Review of Research Design Effects on Response Rates to Questionnaires , 1983 .

[8]  Barbara Snell Dohrenwend.  AN EXPERIMENTAL STUDY OF PAYMENTS TO RESPONDENTS , 1970 .

[9]  D. Kanouse,et al.  Physician Response to a Mailed Survey: An Experiment in Timing of Payment , 1987 .

[10]  W. Fairweather,et al.  Physician response rates to mail and personal interview surveys. , 1979, Public opinion quarterly.

[11]  E. P. Cox.  A Cost/Benefit View of Prepaid Monetary Incentives in Mail Questionnaires , 1976 .

[12]  R. A. Robinson,et al.  Making Mail Surveys More Reliable , 1951 .

[13]  G. Gordon,et al.  Maximizing Returns on Mail Questionnaires , 1958 .

[14]  Lewis R. Aiken,et al.  The Problem of Nonresponse in Survey Research. , 1988 .

[15]  J. Scott Armstrong,et al.  Return Postage in Mail Surveys: A Meta Analysis , 1987 .

[16]  Norman Cliff,et al.  Analyzing Multivariate Data , 1987 .

[17]  Paul B. Duby,et al.  An Evaluation of the Use of Monetary Incentives in Postsecondary Survey Research. , 1987 .

[18]  T. Heberlein,et al.  Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. , 1978 .

[19]  A. S. Linsky.  STIMULATING RESPONSES TO MAILED QUESTIONNAIRES: A REVIEW , 1975 .

[20]  A. Greenberg,et al.  On the Reliability of Mail Questionnaires in Product Tests , 1957 .

[21]  Frederick Wiseman,et al.  Factor Interaction Effects in Mail Survey Response Rates , 1973 .

[22]  J. Armstrong.  Monetary Incentives in Mail Surveys , 1975 .

[23]  Joseph C. Bevis.  ECONOMICAL INCENTIVE USED FOR MAIL QUESTIONNAIRE , 1948 .

[24]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[25]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[26]  J. Goyder,et al.  Further Evidence on Factors Affecting Response Rates to Mailed Questionnaires , 1982 .

[27]  D. Alwin,et al.  CONTINUITY AND CHANGE IN METHODS OF SURVEY DATA ANALYSIS , 1987 .

[28]  Neil M. Ford,et al.  Broadening the Scope of Methodological Research on Mail Surveys , 1976 .

[29]  D. Lockhart,et al.  Making effective use of mailed questionnaires , 1986 .

[30]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[31]  Robert M. Groves,et al.  RESEARCH ON SURVEY DATA QUALITY , 1987 .

[32]  Richard A. Guzzo,et al.  The Effects of Psychologically Based Intervention Programs on Worker Productivity: A Meta-Analysis. , 1985 .

[33]  L. Kanuk,et al.  Mail Surveys and Response Rates: A Literature Review , 1975 .

[34]  R. Stevenson,et al.  Increasing Returns in International Mail Surveys , 1974 .

[35]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .