Accuracy of different modalities of reaction time testing: Implications for online cognitive assessment tools

Reaction time testing is widely used in computerized cognitive assessments, and clinical studies have repeatedly shown it to be a sensitive indicator of cognitive function. Typically, the reaction time test is administered by presenting a subject with a visual stimulus on a computer monitor and prompting the individual to respond (via keypad or computer mouse) as quickly as possible. The individual’s reaction time is calculated as the interval between presentation of the stimulus and the time recorded from the mechanical response. However, there are many inherent latencies and variabilities that may be introduced to the measure by both hardware (computer monitor and mouse) and software (operating system). Because of these delays, we hypothesized that a comparison of hardware protocols (excluding human response) would demonstrate significant differences in the resulting reaction time measures. To simulate the delays of various components of the common systems used to obtain reaction time, we conducted a simple experiment in which either a visual or tactile stimulus evoked a movement from a mechanical transducer to respond to a computer peripheral or a dedicated response device. In the first condition, a simulated visual reaction time test was conducted by flashing a visual stimulus on a computer monitor. The stimulus was detected by a dedicated light sensor, and a linear actuator delivered the mechanical response via computer mouse. The second test condition employed a mobile device as the medium for the visual stimulus, and the mechanical response was delivered to the mobile device’s touchscreen. The third and fourth test conditions simulated tactile reaction time tests in which the stimulus was generated by a dedicated hardware device. The third condition simulated a tactile stimulus, which was detected by a mechanical switch, and again a hardware device delivered the response via computer mouse. The fourth condition also simulated a tactile stimulus, but the response was delivered by a dedicated hardware device designed to store the interval between stimulus delivery and stimulus response. There were significant differences in the range of responses recorded from the four different conditions with the reaction time collected from a visual stimulus on a mobile device being the worst and the device with dedicated hardware designed for the task being the best. The results suggest that some of the commonly used visual tasks on consumer grade computers could be introducing significant errors for reaction time testing and that dedicated hardware designed for the reaction time task is needed to minimize testing errors.

[1]  U. Kujala,et al.  The relation of low grade mental ability to fractures in young men , 2004, International Orthopaedics.

[2]  W. Cole,et al.  Intraindividual Cognitive Variability: An Examination of ANAM4 TBI-MIL Simple Reaction Time Data from Service Members with and without Mild Traumatic Brain Injury , 2017, Journal of the International Neuropsychological Society.

[3]  L. Erasmus,et al.  Multisensory integration after traumatic brain injury: a reaction time study between pairings of vision, touch and audition , 2003, Brain injury.

[4]  D. Stuss,et al.  Reaction time after head injury: fatigue, divided and focused attention, and consistency of performance. , 1989, Journal of neurology, neurosurgery, and psychiatry.

[5]  Tom Whitehouse,et al.  Toward an Experimental Timing Standards Lab: Benchmarking precision in the real world , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[6]  Mark Tommerdahl,et al.  Impaired tactile processing in children with autism spectrum disorder. , 2014, Journal of neurophysiology.

[7]  N. Kleitman,et al.  THE EFFECT OF BODY TEMPERATURE ON REACTION TIME , 1938 .

[8]  Rolf Ulrich,et al.  Time resolution of clocks: Effects on reaction time measurement—Good news for bad clocks , 1989 .

[9]  M. Vogel-Sprott,et al.  Acute dose of alcohol affects cognitive components of reaction time to an omitted stimulus: differences among sensory systems , 2005, Psychopharmacology.

[10]  Judy Shum,et al.  BioMedical Engineering OnLine BioMed Central , 2005 .

[11]  Jörg Beringer,et al.  Timing accuracy of mouse response registration on the IBM microcomputer family , 1992 .

[12]  Youkeun K. Oh,et al.  Reliability and Criterion Validity of a Novel Clinical Test of Simple and Complex Reaction Time in Athletes , 2015, Perceptual and motor skills.

[13]  J. Nijenhuis,et al.  Were the Victorians cleverer than us? The decline in general intelligence estimated from a meta-analysis of the slowing of simple reaction time , 2013 .

[14]  A. Krassioukov,et al.  Multimodality intraoperative monitoring during complex lumbosacral procedures: indications, techniques, and long-term follow-up review of 61 consecutive cases. , 2004, Journal of neurosurgery. Spine.

[15]  L. Arab,et al.  A cross sectional study of the association between walnut consumption and cognitive function among adult us populations represented in NHANES , 2015, The journal of nutrition, health & aging.

[16]  L. Hawk,et al.  Reaction Time Variability in ADHD: A Review , 2012, Neurotherapeutics.

[17]  F Richard Ferraro No evidence of reaction time slowing in autism spectrum disorder , 2016, Autism : the international journal of research and practice.

[18]  A. Benton,et al.  Interactive effects of age and brain disease on reaction time. , 1977, Archives of neurology.

[19]  R. H. Seashore,et al.  Group factors in simple and discriminative reaction times , 1941 .

[20]  J. H. Westerlund,et al.  Relationship between Running Events in Track and Reaction Time , 2013 .

[21]  F. Donders On the speed of mental processes. , 1969, Acta psychologica.

[22]  K. Ciuffreda,et al.  The effect of retinal defocus on simple eye-hand and eye-foot reaction time in traumatic brain injury (TBI) , 2013, Brain injury.

[23]  Y. Lajoie,et al.  Predicting falls within the elderly community: comparison of postural sway, reaction time, the Berg balance scale and the Activities-specific Balance Confidence (ABC) scale for comparing fallers and non-fallers. , 2004, Archives of gerontology and geriatrics.

[24]  Shweta Patil,et al.  Auditory and Visual Reaction Time - A Tool for Early Detection of Neuropathy in Diabetics. - , 2015 .

[25]  Bruce D. McCandliss,et al.  Extent of Microstructural White Matter Injury in Postconcussive Syndrome Correlates with Impaired Cognitive Reaction Time: A 3T Diffusion Tensor Imaging Study of Mild Traumatic Brain Injury , 2008, American Journal of Neuroradiology.

[26]  I. Deary,et al.  Age and sex differences in reaction time in adulthood: results from the United Kingdom Health and Lifestyle Survey. , 2006, Psychology and aging.

[27]  M. Keifer,et al.  Comparison of performance from three continents on the WHO-Recommended Neurobehavioral Core Test Battery. , 1993, Environmental research.

[28]  T. Tombaugh,et al.  Detecting simulation of attention deficits using reaction time tests. , 2006, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[29]  H. Christensen,et al.  Gender Differences in Cognitive Abilities: The Mediating Role of Health State and Health Habits. , 2004 .

[30]  B. Deelman,et al.  Differential effects of simple and choice reaction after closed head injury , 1976, Clinical Neurology and Neurosurgery.

[31]  A. Benton,et al.  Simple and choice reaction time in cerebral disease. , 1955, Confinia neurologica.

[32]  H. Poustchi,et al.  Impact of Hepatitis C Virus Infection on Cognitive Function in Patients With Covert Hepatic Encephalopathy , 2015, Hepatitis monthly.

[33]  M. Corsi-Cabrera,et al.  Effect of total sleep deprivation on reaction time and waking EEG activity in man. , 1995, Sleep.

[34]  D. Stuss,et al.  Reaction time and variability 5 and 10 years after traumatic brain injury. , 1996, Brain injury.

[35]  Daniel J. Wigdor,et al.  Designing for low-latency direct-touch input , 2012, UIST.

[36]  T. Zahn,et al.  Reaction time indicators of attention deficits in closed head injury. , 1999, Journal of clinical and experimental neuropsychology.

[37]  R. H. Burpee,et al.  Measuring Reaction Time of Athletes , 1936 .

[38]  J. A. Leonard Tactual Choice Reactions: I , 1959 .

[39]  I. Deary,et al.  Reaction Time, Age, and Cognitive Ability: Longitudinal Findings from Age 16 to 63 Years in Representative Population Samples , 2005 .

[40]  Ian Neath,et al.  Response time accuracy in Apple Macintosh computers , 2011, Behavior research methods.

[41]  J. M. Cattell,et al.  THE INFLUENCE OF THE INTENSITY OF THE STIMULUS ON THE LENGTH OF THE REACTION TIME , 1886 .

[42]  A. Edwards,et al.  Visual illusion, tactile sensibility and reaction time under LSD-25 , 2004, Psychopharmacologia.

[43]  David McIntyre,et al.  Psychomotor speed in hypertension: effects of reaction time components, stimulus modality, and phase of the cardiac cycle. , 2007, Psychophysiology.

[44]  J. Merkel Die zeitlichen Verhältnisse der Willensthätigkeit , 1883 .

[45]  P. Hancock,et al.  Age differences and changes in reaction time: the Baltimore Longitudinal Study of Aging. , 1994, Journal of gerontology.

[46]  C. Begley,et al.  Drug development: Raise standards for preclinical cancer research , 2012, Nature.

[47]  Jon Driver,et al.  Inhibition of return is supramodal: a demonstration between all possible pairings of vision, touch, and audition , 2000, Experimental Brain Research.

[48]  Nick Hammond,et al.  Self-validating presentation and response timing in cognitive paradigms: How and why? , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[49]  O. Collignon,et al.  Further evidence that congenitally blind participants react faster to auditory and tactile spatial targets. , 2009, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[50]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.

[51]  W. E. Hick Quarterly Journal of Experimental Psychology , 1948, Nature.

[52]  J. Cattell,et al.  V.—MENTAL TESTS AND MEASUREMENTS , 1890 .

[53]  J. Touchon,et al.  Non-degenerative mild cognitive impairment in elderly people and use of anticholinergic drugs: longitudinal cohort study , 2006, BMJ : British Medical Journal.

[54]  Andrew P. Smith,et al.  The effects of idazoxan on reaction times, eye movements and the mood of healthy volunteers and patients with upper respiratory tract illnesses , 1999, Journal of psychopharmacology.

[55]  R. Arciero,et al.  Persistent prolongation of simple reaction time in sports concussion , 2001, Neurology.

[56]  G. Macflynn,et al.  Measurement of reaction time following minor head injury. , 1984, Journal of neurology, neurosurgery, and psychiatry.

[57]  F. Donders,et al.  Over de snelheid van psychische Processen , 1868 .

[58]  J. Siegel,et al.  Reaction-time behaviour as a function of internal-external control of reinforcement and control of test administration. , 1970 .

[59]  Reaction-Time and Motor Development , 1937 .

[60]  R. Hyman Stimulus information as a determinant of reaction time. , 1953, Journal of experimental psychology.

[61]  Biswas Ashoke Kumar,et al.  Reaction time with respect to the nature of stimulus and age of male subjects. , 2010 .

[62]  Richard R Plant,et al.  Millisecond precision psychological research in a world of commodity computers: New hardware, new problems? , 2009, Behavior research methods.

[63]  H. Thompson,et al.  The Mental Traits of Sex: An Experimental Investigation of the Normal Mind, in Men and Women , 2009 .

[64]  C. J. Long,et al.  Visual reaction time and its relationship to neuropsychological test performance. , 1996, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[65]  G. Forbes,et al.  The effect of certain variables on visual and auditory reaction times. , 1945 .

[66]  J. Sergeant,et al.  A motor presetting study in hyperactive, learning disabled and control children. , 1992, Journal of child psychology and psychiatry, and allied disciplines.

[67]  E V Evarts,et al.  Reaction time in Parkinson's disease. , 1981, Brain : a journal of neurology.

[68]  Andrew M. Johnson,et al.  Sex difference in brain nerve conduction velocity in normal humans , 2004, Neuropsychologia.

[69]  G. R. Wells The influence of stimulus duration on reaction time. , 1913 .

[70]  A. Carr,et al.  Reaction time impairment in schizophrenia and affective illness: The role of attention , 1989, Biological Psychiatry.

[71]  Grant L Iverson,et al.  On-Field Predictors of Neuropsychological and Symptom Deficit Following Sports-related Concussion , 2003, Clinical journal of sport medicine : official journal of the Canadian Academy of Sport Medicine.

[72]  J. Ruesch Dark Adaptation, Negative After Images, Tachistoscopic Examinations and Reaction Time in Head Injuries , 1944 .

[73]  W. Schilling,et al.  The Effect of Caffein and Acetanilid on Simple Reaction Time. , 1921 .

[74]  J. Ponsford,et al.  Attentional deficits following closed-head injury. , 1992, Journal of clinical and experimental neuropsychology.

[75]  Sherwood De,et al.  Cardiorespiratory Health, Reaction Time and Aging , 1979 .

[76]  D. Sherwood,et al.  Cardiorespiratory health, reaction time and aging. , 1979, Medicine and science in sports.

[77]  Tom Whitehouse,et al.  How choice of mouse may affect response timing in psychological studies , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[78]  Andrew P. Smith,et al.  Effects of caffeine on mood and performance: a study of realistic consumption , 2002, Psychopharmacology.

[79]  E. T. Klemmer Simple reaction time as a function of time uncertainty. , 1957, Journal of experimental psychology.

[80]  F. Goodenough,et al.  The development of the reactive process from early childhood to maturity. , 1935 .

[81]  L. Henderson,et al.  On the existence of an attention-demanding process peculiar to simple reaction time: Converging evidence from Parkinson's disease , 1989 .

[82]  R. Davidson,et al.  Reaction time measures of interhemispheric transfer time in reading disabled and normal children , 1990, Neuropsychologia.

[83]  Richard R Plant,et al.  A reminder on millisecond timing accuracy and potential replication failure in computer-based psychology experiments: An open letter , 2016, Behavior research methods.