Assessing intraindividual variability in sustained attention: reliability, relation to speed and accuracy, and practice effects

We investigated the psychometric properties of competing measures of sustained attention. 179 subjects were assessed twice within seven day's time with a test designed to measure sustained atten- tion, or concentration, respectively. In addition to traditional performance indices (i.e., speed (MRT) and accuracy (E%)), we evaluated two intraindividual response time (RT) variability measures: standard deviation (SDRT) and coefficient of variation (CVRT). For the overall test, both indices were reliable. SDRT showed good to acceptable retest reliability for all subtests. For CVRT, retest reliability coeffi- cients ranged from very good to not satisfactory. While the reversed-word recognition test proved highly reliable, the mental calculation test and the arrows test were not sufficiently reliable. CVRT was only slightly correlated but SDRT was highly correlated with MRT. In contrast to substantial practice gains for MRT, SDRT and E%, only CVRT proved to be stable. In conclusion, CVRT appears to be a poten- tial index for assessing performance variability: it is reliable for the overall test, only moderately corre- lated with speed, and virtually not affected by practice. However, before applying CVRT in practical assessment settings, additional research is required to elucidate the impact of task-specific factors on the reliability of this performance measure.

[1]  Rolf Ulrich,et al.  Testing the race model inequality: An algorithm and computer programs , 2007, Behavior research methods.

[2]  Rolf Ulrich,et al.  Central Slowing During the Night , 2007, Psychological science.

[3]  P. Maruff,et al.  Practice Effects Associated with the Repeated Assessment of Cognitive Function Using the CogState Battery at 10-minute, One Week and One Month Test-retest Intervals , 2006, Journal of clinical and experimental neuropsychology.

[4]  H. Smid,et al.  Sustained and Transient Attention in the Continuous Performance Task , 2006, Journal of clinical and experimental neuropsychology.

[5]  Kristina M. Visscher,et al.  The neural bases of momentary lapses in attention , 2006, Nature Neuroscience.

[6]  S. MacDonald,et al.  Intraindividual Variability in Vigilance Performance: Does Degrading Visual Stimuli Mimic Age-Related “Neural Noise”? , 2006, Journal of clinical and experimental neuropsychology.

[7]  C. Anderson,et al.  Sleepiness enhances distraction during a monotonous task. , 2006, Sleep.

[8]  E. Strauss,et al.  Intraindividual Variability as a Marker of Neurological Dysfunction: A Comparison of Alzheimer's Disease and Parkinson's Disease , 2006, Journal of clinical and experimental neuropsychology.

[9]  Ian H. Robertson,et al.  Dissecting the attention deficit hyperactivity disorder (ADHD) phenotype: Sustained attention, response variability and spatial attentional asymmetries in relation to dopamine transporter (DAT1) genotype , 2005, Neuropsychologia.

[10]  T Shallice,et al.  Impaired concentration due to frontal lobe damage from two distinct lesion sites , 2005, Neurology.

[11]  Michael D. Robinson,et al.  Neuroticism as mental noise: a relation between neuroticism and reaction time standard deviations. , 2005, Journal of personality and social psychology.

[12]  Peter C. M. Molenaar,et al.  On the relation between the mean and the variance of a diffusion model response time distribution , 2005 .

[13]  F. Castellanos,et al.  Varieties of Attention-Deficit/Hyperactivity Disorder-Related Intra-Individual Variability , 2005, Biological Psychiatry.

[14]  H. Christensen,et al.  Within-occasion intraindividual variability and preclinical diagnostic status: is intraindividual variability an indicator of mild cognitive impairment? , 2005, Neuropsychology.

[15]  K Richard Ridderinkhof,et al.  Delta plots in the study of individual differences: new tools reveal response inhibition deficits in AD/Hd that are eliminated by methylphenidate treatment. , 2005, Journal of abnormal psychology.

[16]  Are Working Memory and Attention Related Constructs , 2005 .

[17]  S. MacDonald,et al.  Inconsistency in serial choice decision and motor reaction times dissociate in younger and older adults , 2004, Brain and Cognition.

[18]  Jonathan Smallwood,et al.  Subjective experience and the attentional lapse: Task engagement and disengagement during sustained attention , 2004, Consciousness and Cognition.

[19]  James A. Kole,et al.  Effects of prolonged work on data entry speed and accuracy. , 2004, Journal of experimental psychology. Applied.

[20]  Helfried Moosbrugger,et al.  Attention and Working Memory as Predictors of Intelligence. , 2004 .

[21]  S. Urbina.  Essentials of Psychological Testing , 2005, PsyPag Quarterly.

[22]  D. Stuss,et al.  Staying on the job: the frontal lobes control individual performance variability. , 2003, Brain : a journal of neurology.

[23]  D. Friedman.  Cognition and Aging: A Highly Selective Overview of Event-Related Potential (ERP) Data , 2003, Journal of clinical and experimental neuropsychology.

[24]  Konstruktion eines Komplexen Konzentrationstests , 2003 .

[25]  P. Maruff,et al.  Behavioral Characterization of Mild Cognitive Impairment , 2002, Journal of clinical and experimental neuropsychology.

[26]  J. Arruda,et al.  Measures of Variations in Performance During a Sustained Attention Task , 2002, Journal of clinical and experimental neuropsychology.

[27]  F. Craik,et al.  Lapses of Intention and Performance Variability Reveal Age-Related Increases in Fluctuations of Executive Control , 2002, Brain and Cognition.

[28]  S. MacDonald,et al.  Variability in reaction time performance of younger and older adults. , 2002, The journals of gerontology. Series B, Psychological sciences and social sciences.

[29]  P. Rabbitt,et al.  There are stable individual differences in performance variability, both from moment to moment and from day to day , 2001, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[30]  J. C. Ballard.  Assessing Attention: Comparison of Response-Inhibition and Traditional Continuous Performance Tests , 2001, Journal of clinical and experimental neuropsychology.

[31]  G. Matthews.  Human Performance: Cognition, Stress and Individual Differences , 2000 .

[32]  S. MacDonald,et al.  Intraindividual variability in cognitive performance in older adults: comparison of adults with mild dementia, adults with arthritis, and healthy adults. , 2000, Neuropsychology.

[33]  P. Graf,et al.  Exacting Assessments: Do Older Adults Fatigue More Quickly? , 2000, Journal of clinical and experimental neuropsychology.

[34]  C Leth-Steensen,et al.  Mean response times, variability, and skew in the responding of ADHD children: a response time distributional approach. , 2000, Acta psychologica.

[35]  T. Zahn,et al.  Reaction time indicators of attention deficits in closed head injury. , 1999, Journal of clinical and experimental neuropsychology.

[36]  S. Segalowitz,et al.  RT coefficient of variation is differentially sensitive to executive control involvement in an attention switching task , 1999 .

[37]  Karl M. Newell,et al.  Is Variability in Human Performance a Reflection of System Noise? , 1998 .

[38]  I. Reinvang.  Validation of reaction time in continuous performance tasks as an index of attention by electrophysiological measures. , 1998, Journal of clinical and experimental neuropsychology.

[39]  Donald T. Stuss,et al.  Aging and Variability in Performance , 1998 .

[40]  J. Rapoport,et al.  Attention deficits in childhood-onset schizophrenia: reaction time studies. , 1998, Journal of abnormal psychology.

[41]  A. Jensen,et al.  The g factor , 1996, Nature.

[42]  T. Monk,et al.  Speed of mental processing in the middle of the night. , 1997, Sleep.

[43]  S. Segalowitz,et al.  Attentional factors in response time variability after traumatic brain injury: An ERP study , 1997, Journal of the International Neuropsychological Society.

[44]  D. Stuss,et al.  Do long tests yield a more accurate diagnosis of dementia than short tests? A comparison of 5 neuropsychological tests. , 1996, Archives of neurology.

[45]  A. V. D. Ven,et al.  Inhibition in speed and concentration tests: the Poisson inhibition model , 1995 .

[46]  John Whyte,et al.  Sustained arousal and attention after traumatic brain injury , 1995, Neuropsychologia.

[47]  Donald T. Stuss,et al.  Characterization of stability of performance in patients with traumatic brain injury: Variability and consistency on reaction time tests. , 1994 .

[48]  M. Ron,et al.  Effects of practice of serial tests of attention in healthy subjects. , 1994, Journal of clinical and experimental neuropsychology.

[49]  R. Ulrich,et al.  Effects of truncation on reaction time analysis. , 1994, Journal of experimental psychology. General.

[50]  Jeff Miller,et al.  Information processing models generating lognormally distributed reaction times , 1993 .

[51]  Sidney J. Segalowitz,et al.  Skilled performance, practice, and the differentiation of speed-up from automatization effects: Evidence from second language word recognition , 1993, Applied Psycholinguistics.

[52]  T. Cochrane,et al.  Blocks in choice responding as a function of age and physical fitness. , 1993, Psychology and aging.

[53]  G. Logan.  Shapes of reaction-time distributions and shapes of learning curves: a test of the instance theory of automaticity. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[54]  A. Jensen,et al.  THE IMPORTANCE OF INTRAINDIVIDUAL VARIATION IN REACTION TIME , 1992 .

[55]  P M Rabbitt,et al.  Effects of Alcohol, Practice, and Task Complexity on Reaction Time Distributions , 1992, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[56]  Gerald E. Larson,et al.  Reaction time variability and intelligence: A “worst performance” analysis of individual differences , 1990 .

[57]  G Salvendy,et al.  Microbreak length, performance, and stress in a data entry task. , 1989, Ergonomics.

[58]  A. Carr,et al.  Reaction time impairment in schizophrenia and affective illness: The role of attention , 1989, Biological Psychiatry.

[59]  P. Rabbitt,et al.  How does very prolonged practice improve decision speed , 1989 .

[60]  G. Hockey.  Changes in operator efficiency as a function of environmental stress, fatigue, and circadian rhythms. , 1986 .

[61]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[62]  J. Pieters.  Reaction time analysis of simple mental tasks: A general approach , 1985 .

[63]  R. Hockey.  Stress and fatigue in human performance , 1984 .

[64]  A. T. Welford,et al.  Relationships between reaction time and fatigue, stress, age and sex , 1980 .

[65]  W. Surwillo.  Reaction-time variability, periodicities in reaction-time distributions, and the EEG gating-signal hypothesis , 1975, Biological Psychology.

[66]  Stephen J. Boies,et al.  Components of attention. , 1971 .

[67]  A. Sanders,et al.  On the effects of continuous active work on performance , 1970 .

[68]  G. Kellas,et al.  Distribution of reaction times of retardates and normals. , 1968, American journal of mental deficiency.

[69]  G. Berkson,et al.  Reaction time variability of mental defectives and normals. , 1967, American journal of mental deficiency.

[70]  W. S. Appleton.  Concentration: The Phenomenon and Its Disruption , 1967 .

[71]  Paul Bertelson,et al.  Blockings in Prolonged Serial Responding , 1963 .

[72]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[73]  D W FISKE,et al.  Intra-individual response variability. , 1955, Psychological bulletin.

[74]  W. Holtzman.  Fundamental statistics in psychology and education. , 1951 .

[75]  C. Buxton,et al.  The Psychology of Efficiency. , 1944 .

[76]  J. Guilford,et al.  Fundamental statistics in psychology and education , 1943 .

[77]  A. G. Bills.  Facilitation and inhibition in mental work. , 1937 .

[78]  A. G. Bills.  Some causal factors in mental blocking. , 1935 .

[79]  A. G. Bills.  Blocking: a new principle of mental fatigue. , 1931 .

[80]  C. Spearman,et al.  "THE ABILITIES OF MAN". , 1928, Science.

[81]  J. Guilford.  'Fluctuations of attention' with weak visual stumuli , 1927 .

[82]  E. Boring,et al.  The factor of speed in intelligence , 1926 .

[83]  E. S. Robinson,et al.  Two factors in the work decrement , 2022 .

[84]  H. Woodrow.  The measurement of attention , 1914 .

[85]  H. Obersteiner.  EXPERIMENTAL RESEARCHES ON ATTENTION , 1879 .

[86]  C. McDiarmid.  Concentration , 1862, The Dental register.