Pixels or pencils? The relative effectiveness of Web-based versus paper surveys

A survey of academic reference librarians provided data for an examination of differences in survey administration on paper and the World Wide Web, with particular attention to response rates, response time/quickness, sampling bias, and differences in responses that are related to the medium of administration. Three methods of survey distribution were compared: paper survey distributed by mail, Web survey announced by mail, and Web survey announced by e-mail. Response rates for the Web survey were not as high as with traditional survey methods. Responses to the Web survey were returned more quickly, but not by a great deal. E-mail notices were more effective for promoting the Web survey than paper notices. The study detected no sampling bias or differences in the content of responses. Although the advantages of Web survey administration are alluring, researchers must use carefully designed follow-up procedures and avoid technology-related implementation problems to match the effectiveness of traditional paper surveys.

[1]  Paul Rosenfeld,et al.  Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? , 1992 .

[2]  Merrill L. Meehan,et al.  E-Mail Survey of a Listserv Discussion Group: Lessons Learned from Surveying an Electronic Network of Learners. , 1997 .

[3]  D. Paulhus.  Two-component models of socially desirable responding. , 1984 .

[4]  Ann Roselle,et al.  The utility of electronic mail follow-ups for library research , 1998 .

[5]  Nicolaos E. Synodinos,et al.  Computer-administered versus paper-and-pencil surveys and the effect of sample selection , 1994 .

[6]  Christine B. Smith,et al.  Casting the Net: Surveying an Internet Population , 2006, J. Comput. Mediat. Commun..

[7]  Raj Mehta,et al.  Comparing Response Rates and Response Content in Mail versus Electronic Mail Surveys , 1995 .

[8]  John W. Daly,et al.  Electronic bulletin board distributed questionnaires for exploratory research , 1996, J. Inf. Sci..

[9]  Charles H. Davis,et al.  The Modern Language Association: electronic and paper surveys of computer-based tool use , 1996 .

[10]  H A Skinner,et al.  Does the computer make a difference? Computerized versus face-to-face versus self-report assessment of alcohol, drug, and tobacco use. , 1983, Journal of consulting and clinical psychology.

[11]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .

[12]  Judith S. Kaufman,et al.  Factors influencing therapist selection among gays, lesbians and bisexuals , 1997 .

[13]  Yin Zhang.  Using the Internet for survey research: a case study , 2000 .

[14]  S. Kiesler,et al.  Response Effects in the Electronic Survey , 1986 .

[15]  James E. Pitkow,et al.  Emerging trends in the WWW user population , 1996, CACM.

[16]  Andrew N. Williams,et al.  Using electronic mail to assess undergraduates’ experiences: Assessing frame and mode effects , 1998, J. Comput. High. Educ..

[17]  John Carlo Bertot,et al.  Electronic Surveys: Methodological Implications for Using the World Wide Web to Collect Survey Data , 1996 .

[18]  S. Kiesler,et al.  SELF-SELECTED AND RANDOMLY SELECTED RESPONDENTS IN A COMPUTER NETWORK SURVEY , 1992 .

[19]  Lee Sproull,et al.  Using Electronic Mail for Data Collection in Organizational Research , 1986 .

[20]  C. Michele Matz,et al.  Administration of Web versus Paper Surveys: Mode Effects and Response Rates. , 1999 .

[21]  R. Weitkunat,et al.  Social Science Computer Review Internet Surveys by Direct Mailing : an Innovative Way of Collecting Data Internet Surveys by Direct Mailing an Innovative Way of Collecting Data , 2022 .

[22]  Earl R. Babbie,et al.  The practice of social research , 1969 .

[23]  J. Greist,et al.  The reliability of a computer interview for drug use/abuse information , 1983 .

[24]  Zane L. Berge,et al.  IPCT Journal readership survey , 1996 .

[25]  Susan E. Anderson,et al.  Using Electronic Mail Surveys and Computer-Monitored Data for Studying Computer-Mediated Communication Systems , 1995 .

[26]  Michael Margolis,et al.  Breaking ground on the virtual frontier: Surveying civic life on the internet , 1996 .

[27]  James G. Helgeson,et al.  The Decision Process Equivalency of Electronic Versus Pencil-and-Paper Data Collection Methods , 1989 .