Psychophysics in a Web browser? Comparing response times collected with JavaScript and Psychophysics Toolbox in a visual search task

Behavioral researchers are increasingly using Web-based software such as JavaScript to conduct response time experiments. Although there has been some research on the accuracy and reliability of response time measurements collected using JavaScript, it remains unclear how well this method performs relative to standard laboratory software in psychologically relevant experimental manipulations. Here we present results from a visual search experiment in which we measured response time distributions with both Psychophysics Toolbox (PTB) and JavaScript. We developed a methodology that allowed us to simultaneously run the visual search experiment with both systems, interleaving trials between two independent computers, thus minimizing the effects of factors other than the experimental software. The response times measured by JavaScript were approximately 25 ms longer than those measured by PTB. However, we found no reliable difference in the variability of the distributions related to the software, and both software packages were equally sensitive to changes in the response times as a result of the experimental manipulations. We concluded that JavaScript is a suitable tool for measuring response times in behavioral research.

[1]  Joshua de Leeuw,et al.  jsPsych: A JavaScript library for creating behavioral experiments in a Web browser , 2014, Behavior Research Methods.

[2]  Robert X D Hawkins,et al.  Conducting real-time multiplayer experiments on the web , 2014, Behavior Research Methods.

[3]  Travis Simcox,et al.  Collecting response times using Amazon Mechanical Turk and Adobe Flash , 2013, Behavior Research Methods.

[4]  G. Francis.  Replication, statistical consistency, and publication bias. , 2013 .

[5]  Bill Tomlinson,et al.  Who are the crowdworkers?: shifting demographics in mechanical turk , 2010, CHI Extended Abstracts.

[6]  E. Reingold,et al.  Visual search asymmetry: The influence of stimulus familiarity and low-level features , 2001, Perception & psychophysics.

[7]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[8]  John K Kruschke,et al.  Posterior predictive checks can and should be Bayesian: comment on Gelman and Shalizi, 'Philosophy and the practice of Bayesian statistics'. , 2013, The British journal of mathematical and statistical psychology.

[9]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[10]  Neil Stewart,et al.  Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments , 2014, Behavior research methods.

[11]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[12]  Richard R Plant,et al.  Millisecond precision psychological research in a world of commodity computers: New hardware, new problems? , 2009, Behavior research methods.

[13]  J. Wolfe,et al.  What Can 1 Million Trials Tell Us About Visual Search? , 1998 .

[14]  J. Arnett.  The neglected 95%: why American psychology needs to become less American. , 2008, The American psychologist.

[15]  Ian Neath,et al.  Response time accuracy in Apple Macintosh computers , 2011, Behavior research methods.

[16]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[17]  S. Levinson,et al.  WEIRD languages have misled us, too , 2010, Behavioral and Brain Sciences.

[18]  Ulf-Dietrich Reips,et al.  A Brief History of Web Experimenting , 2000 .

[19]  J. Kruschke.  Doing Bayesian Data Analysis: A Tutorial with R and BUGS , 2010 .

[20]  C. B. Colby.  The weirdest people in the world , 1973 .

[21]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[22]  Diniz Lopes,et al.  ScriptingRT: A Software Library for Collecting Response Latencies in Online Studies of Cognition , 2013, PloS one.

[23]  Diego López-de-Ipiña,et al.  Presentation Accuracy of the Web Revisited: Animation Methods in the HTML5 Era , 2014, PloS one.

[24]  Matthew J. Denwood,et al.  runjags: An R Package Providing Interface Utilities, Model Templates, Parallel Computing Methods and Additional Distributions for MCMC Models in JAGS , 2016 .

[25]  P Cavanagh,et al.  Familiarity and pop-out in visual search , 1994, Perception & psychophysics.

[26]  Cosma Rohilla Shalizi,et al.  Philosophy and the practice of Bayesian statistics. , 2010, The British journal of mathematical and statistical psychology.

[27]  Neil Stewart,et al.  Using Adobe Flash Lite on mobile phones for psychological research: Reaction time measurement reliability and interdevice variability , 2008, Behavior research methods.

[28]  Neil Stewart,et al.  Adobe Flash as a medium for online experimentation: A test of reaction time measurement capabilities , 2007, Behavior research methods.

[29]  Rolf A. Zwaan,et al.  Revisiting Mental Simulation in Language Comprehension: Six Replication Attempts , 2012, PloS one.

[30]  Todd M. Gureckis,et al.  CUNY Academic , 2016 .

[31]  Brian A. Nosek,et al.  Harvesting implicit group attitudes and beliefs from a demonstration web site , 2002 .

[32]  Armand Clercq,et al.  A simple and sensitive method to measure timing accuracy , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[33]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[34]  Matthew R. Yanko,et al.  LCDs are better: Psychophysical and photometric estimates of the temporal characteristics of CRT and LCD monitors , 2012, Attention, Perception, & Psychophysics.

[35]  Erwin Haasnoot,et al.  QRTEngine: An easy solution for running online reaction time experiments using Qualtrics , 2014, Behavior research methods.