Testing differential effects of computer-based, web-based and paper-based administration of questionnaire research instruments

Translation of questionnaire instruments to digital administration systems, both self-contained and web-based, is widespread and increasing daily. However, the literature is lean on controlled empirical studies investigating the potential for differential effects of administrative methods. In this study, two university student samples were administered 16 questionnaires across three separate administration conditions: paper-based, computer-based and web-based. Outcomes of interest included data quality and participant affect. Overall, few differences in data quality were observed between administration conditions despite some evidence in favour of paper-based administration (PBA) over the other two. Affective responses of participants favoured the PBA over web- and computer-based administrations. Implications for research use of digital systems for data collection are discussed.

[1]  Olle Bälter,et al.  Web-based and Mailed Questionnaires: A Comparison of Response Rates and Compliance , 2005, Epidemiology.

[2]  H. Minnis,et al.  The Strengths and Difficulties Questionnaire , 2003, European Child & Adolescent Psychiatry.

[3]  P Wright,et al.  Computer anxiety: a comparison of pen-based personal digital assistants, conventional computer and paper assessment of mood and performance. , 1998, British journal of psychology.

[4]  M. Diefenbach,et al.  A Multimedia Interactive Education System for Prostate Cancer Patients: Development and Preliminary Evaluation , 2004, Journal of medical Internet research.

[5]  Kate Lorig,et al.  Internet Versus Mailed Questionnaires: A Randomized Comparison , 2004, Journal of medical Internet research.

[6]  Darin J Erickson,et al.  Handheld Computers , 2003, Evaluation review.

[7]  Stefan E. Schulenberg,et al.  The equivalence of computerized and paper-and-pencil psychological instruments: Implications for measures of negative affect , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[8]  Clay E. George,et al.  The effects of computerized versus paper-and-pencil administration on measures of negative affect , 1992 .

[9]  Angela F. L. Wong,et al.  Computer anxiety and its correlates : a meta-analysis , 1999 .

[10]  Patricia L. Hardré,et al.  Production and data management issues for digital questionnaire administration , 2005 .

[11]  Barbara A. Greene,et al.  Engagement in Academic Work: The Role of Learning Goals, Future Consequences, Pleasing Others, and Perceived Ability , 1996, Contemporary educational psychology.

[12]  Roy B. Clariana,et al.  Paper-based versus computer-based assessment: key factors associated with the test mode effect , 2002, Br. J. Educ. Technol..

[13]  Barbara A. Greene,et al.  Distributive Learning in Introductory Chemical Engineering: University Students' Learning, Motivation, and Attitudes Using a CD-ROM , 2003 .

[14]  L.Michael Honaker,et al.  The equivalency of computerized and conventional MMPI administration: A critical review☆☆☆ , 1988 .

[15]  James A. Hall,et al.  A Comparison of Computer-Based Versus Pencil-and-Paper Assessment of Drug Use , 2002 .

[16]  Yin Zhang.  Using the Internet for survey research: a case study , 2000 .

[17]  John F. Marszalek,et al.  Electronic Surveys: Ethical Issues for Researchers. , 1995 .

[18]  Patricia L. Hardré,et al.  A motivational model of rural students' intentions to persist in, versus drop out of, high school , 2003 .

[19]  Don A. Dillman,et al.  Survey Mode as a Source of Instability in Responses across Surveys , 2005 .

[20]  Jane Webster,et al.  Computer-assisted versus paper-and-pencil administration of questionnaires , 1996 .

[21]  Zane L. Berge,et al.  IPCT Journal readership survey , 1996 .

[22]  R. Vallerand,et al.  Self-determination and persistence in a real-life setting: toward a motivational model of high school dropout. , 1997, Journal of personality and social psychology.

[23]  Don A. Dillman,et al.  Presidential Address: Navigating the Rapids of Change: Some Observations on Survey Methodology in the Early Twenty-First Century , 2002 .

[24]  J. S. St. Lawrence,et al.  A comparison of Web- with paper-based surveys of gay and bisexual men who vacationed in a gay resort community. , 2004, AIDS education and prevention : official publication of the International Society for AIDS Education.

[25]  Paul Rosenfeld,et al.  Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? , 1992 .