Timing accuracy of Web experiments: A case study using the WebExp software package

Although Internet-based experiments are gaining in popularity, most studies rely on directly evaluating participants’ responses rather than response times. In the present article, we present two experiments that demonstrate the feasibility of collecting response latency data over the World-Wide Web using WebExp—a software package designed to run psychological experiments over the Internet. Experiment 1 uses WebExp to collect measurements for known time intervals (generated using keyboard repetition). The resulting measurements are found to be accurate across platforms and load conditions. In Experiment 2, we use WebExp to replicate a lab-based self-paced reading study from the psycholinguistic literature. The data of the Web-based replication correlate significantly with those of the original study and show the same main effects and interactions. We conclude that WebExp can be used to obtain reliable response time data, at least for the self-paced reading paradigm.

[1]  B Myors,et al.  Timing accuracy of PC programs running under DOS and Windows , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[2]  Kenneth I Forster,et al.  DMDX: A Windows display program with millisecond accuracy , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[3]  Frank Keller,et al.  Phonology competes with syntax: experimental evidence for the interaction of word order and accent placement in the realization of Information Structure , 2001, Cognition.

[4]  Sean Hammond,et al.  Research methods in psychology, 3rd ed. , 2006 .

[5]  E. B. Zechmeister,et al.  Research Methods in Psychology. , 1990 .

[6]  M. Pickering,et al.  Structural change and reanalysis difficulty in language comprehension , 1999 .

[7]  Neil Stewart,et al.  Adobe Flash as a medium for online experimentation: A test of reaction time measurement capabilities , 2007, Behavior research methods.

[8]  Milton Lodge,et al.  Magnitude Scaling: Quantitative Measurement of Opinions , 1981 .

[9]  Angela Brunstein,et al.  DEWEX: A system for designing and conducting Web-based experiments , 2007, Behavior research methods.

[10]  W C Schmidt Presentation accuracy of Web animation methods , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[11]  Kathleen A. Welsh-Bohmer,et al.  Hardware and software for tachistoscopy: How to make accurate measurements on any PC utilizing the Microsoft Windows operating system , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[12]  John H. Krantz,et al.  Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness , 1997 .

[13]  Jens D M Rademacher,et al.  Dynamic online surveys and experiments with the free open-source softwaredynQuest , 2007, Behavior research methods.

[14]  J. Woolley,et al.  Paradigms and processes in reading comprehension. , 1982, Journal of experimental psychology. General.

[15]  Susan M. Garnsey,et al.  The Contributions of Verb Bias and Plausibility to the Comprehension of Temporarily Ambiguous Sentences , 1997 .

[16]  Ulf-Dietrich Reips,et al.  WEXTOR: A Web-based tool for generating and visualizing experimental designs and procedures , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[17]  M. Corley,et al.  Syntactic priming in English sentence production: Categorical and latency evidence from an Internet-based study , 2002, Psychonomic bulletin & review.

[18]  Subahshini Gunasekharan,et al.  Evaluation of Web Experiments , 2007 .

[19]  W J MacInnes,et al.  Millisecond timing on PCs and Macs , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[20]  M. Masson Using confidence intervals for graphically based data interpretation. , 2003, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[21]  Ulf-Dietrich Reips Standards for Internet-based experimenting. , 2002, Experimental psychology.

[22]  Sam Featherston,et al.  Universals and grammaticality: wh-constraints in German and English , 2005 .

[23]  Jennifer E. Arnold,et al.  The Concomitant Effects of Phrase Length and Informational Content in Sentence Comprehension , 2000, Journal of psycholinguistic research.

[24]  Matthew Flatt,et al.  PsyScope: An interactive graphic system for designing and controlling experiments in the psychology laboratory using Macintosh computers , 1993 .

[25]  J Eichstaedt,et al.  An inaccurate-timing filter for reaction time measurement by JAVA applets implementing Internet-based experiments , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.