A Comparison of Survey Data Collected by Regular Mail and Electronic Mail Questionnaires

A survey was conducted by the National Council on Measurement in Education (NCME) to examine the telecommunications needs of the organization's membership. A component of this study permitted an examination of two response modalities (regular mail and e-mail) across a number of variables. Separate samples of 585 were drawn to take the survey. The first sample consisted of all those organization members who had registered their e-mail address with the organization while the second sample was randomly selected from those members who did not list an e-mail address. The first group was sent a nine question instrument via e-mail while the second group was asked to fill out a ten question instrument via regular mail. Overall response rates were low and significantly different [30% for the e-mail group and 36% for the regular mail group, χ2 (1) = 10.42, p < .01], though not uncommon for institutional surveys. Individual item response rates, however, were statistically significantly higher for the e-mail group across a number of dimensions. The results suggest that for some organizations, e-mail questionnaires may be a viable mechanism for conducting surveys. Suggestions are made on how to improve overall response rates.

[1]  Earl R. Babbie,et al.  Survey Research Methods , 1984 .

[2]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[3]  S. Kiesler,et al.  SELF-SELECTED AND RANDOMLY SELECTED RESPONDENTS IN A COMPUTER NETWORK SURVEY , 1992 .

[4]  W. Fairweather,et al.  Physician response rates to mail and personal interview surveys. , 1979, Public opinion quarterly.

[5]  Robert M. Groves,et al.  Surveys by Telephone: A National Comparison With Personal Interviews. , 1980 .

[6]  S. Kiesler,et al.  Response Effects in the Electronic Survey , 1986 .

[7]  J. H. Frey,et al.  Survey research by telephone , 1983 .

[8]  Linda M. Doherty,et al.  Impression Management, Candor, and Microcomputer-Based Organizational Surveys: An Individual Differences Approach. , 1991 .

[9]  W. Aquilino,et al.  EFFECTS OF INTERVIEW MODE ON SELF-REPORTED DRUG USE , 1990 .

[10]  Christopher L. Martin,et al.  Some effects of computerized interviewing on job applicant responses , 1989 .

[11]  J. Goyder,et al.  Face-to-Face Interviews and Mailed Questionnaires: The Net Difference in Response Rate , 1985 .

[12]  M. Traugott,et al.  THE POLLS—A REVIEW: EXIT POLLS IN THE 1989 VIRGINIA GUBERNATORIAL RACE: WHERE DID THEY GO WRONG? , 1992 .

[13]  Lee Sproull,et al.  Using Electronic Mail for Data Collection in Organizational Research , 1986 .

[14]  Chuck Huff,et al.  The on-line voyeur: Promises and pitfalls of observing electronic interaction , 1989 .

[15]  T. Heberlein,et al.  Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. , 1978 .

[16]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[17]  Floyd J. Fowler,et al.  Survey Research Methods , 1984 .

[18]  Paul Beatty,et al.  RESPONSE RATES AND RESPONSE CONTENT IN MAIL VERSUS FACE-TO-FACE SURVEYS , 1994 .

[19]  Paul Rosenfeld,et al.  Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? , 1992 .