Methods of testing for a deficit in single-case studies: Evaluation of statistical power by Monte Carlo simulation

Testing for the presence of a deficit by comparing a case to controls is a fundamental feature of many neuropsychological single-case studies. Monte Carlo simulation was employed to study the statistical power of two competing approaches to this task. The power to detect a large deficit was low to moderate for a method proposed by Crawford and Howell (1998; ranging from 44% to 63%) but was extremely low for a method proposed by Mycroft, Mitchell, and Kay (2002; ranging from 1% to 13%). The effects of departures from normality were examined, as was the effect of varying degrees of measurement error in the scores of controls and the single case. Measurement error produced a moderate reduction in power when present in both controls and the case; the effect of differentially greater measurement error for the single case depended on the initial level of power. When Mycroft et al.'s method was used to test for the presence of a classical dissociation, it produced very high Type I error rates (ranging from 20.7% to 49.3%); in contrast, the rates for criteria proposed by Crawford and Garthwaite (2005b) were low (ranging from 1.3% to 6.7%). The broader implications of these results for single-case research are discussed.

[1]  F. Schmidt,et al.  Measurement Error in Psychological Research: Lessons From 26 Research Scenarios , 1996 .

[2]  L. T. DeCarlo.  On the meaning and use of kurtosis. , 1997 .

[3]  C. Burt.  Is Intelligence Distributed Normally , 1963 .

[4]  M. Coltheart.  Assumptions and methods in cognitive neuropsychology. , 2001 .

[5]  A. Milner,et al.  Automatic avoidance of obstacles is a dorsal stream function: evidence from optic ataxia , 2004, Nature Neuroscience.

[6]  A. Lamba,et al.  Differential deficits in expression recognition in gene-carriers and patients with Huntington ’ s disease , 2003 .

[7]  Donald B. Owen,et al.  Survey of Properties and Applications of the Noncentral t-Distribution , 1968 .

[8]  P. Garthwaite,et al.  Inferential methods for comparing a single case with a control sample: modified t‐tests versus mycroft et al.'s (2002) modified anova , 2004, Cognitive neuropsychology.

[9]  Dissociation of affective modulation of recollective and perceptual experience following amygdala damage , 2003, Journal of neurology, neurosurgery, and psychiatry.

[10]  Diane Swick,et al.  Orthography Influences the Perception of Speech in Alexic Patients , 2003, Journal of Cognitive Neuroscience.

[11]  K. Priftis,et al.  Arithmetic priming from neglected numbers , 2006, Cognitive neuropsychology.

[12]  M. Jeannerod,et al.  The role of proprioception in action recognition , 2003, Consciousness and Cognition.

[13]  A. Caramazza,et al.  The case for single-patient studies , 1988 .

[14]  P. Garthwaite,et al.  Investigation of the single case in neuropsychology: confidence limits on the abnormality of test scores and test score differences , 2002, Neuropsychologia.

[15]  G. Humphreys,et al.  Visuomotor cuing through tool use in unilateral visual neglect. , 2004, The Journal of general psychology.

[16]  A. Azzalini,et al.  Statistical applications of the multivariate skew normal distribution , 2009, 0911.2093.

[17]  Arjun K. Gupta,et al.  A multivariate skew normal distribution , 2004 .

[18]  P. Garthwaite,et al.  The distribution of the difference between two t-variates and application to a test for dissociations , 2004 .

[19]  D. Mitchell,et al.  Comparing a single case to a control sample: differences in distribution versus differences in means , 2004, Cognitive neuropsychology.

[20]  Keith R. Laws,et al.  Testing for a deficit in single-case studies: Effects of departures from normality , 2006, Neuropsychologia.

[21]  Tim M. Gale,et al.  When is Category Specific in Alzheimer's Disease? , 2005, Cortex.

[22]  Janice Kay,et al.  An evaluation of statistical procedures for comparing an individual's performance with that of a group of controls , 2002, Cognitive neuropsychology.

[23]  A. Caramazza.  On drawing inferences about the structure of normal cognitive systems from the analysis of patterns of impaired performance: The case for single-patient studies , 1986, Brain and Cognition.

[24]  G. Engel,et al.  Neuropsychology , 1994, Schizophrenia Research.

[25]  Paul H Garthwaite,et al.  Testing for suspected impairments and dissociations in single-case studies in neuropsychology: evaluation of alternatives using monte carlo simulations and revised tests for dissociations. , 2005, Neuropsychology.

[26]  I. Deary.  Age-Associated Memory Impairment: A Suitable Case for Treatment? , 1995, Ageing and Society.

[27]  Tim Shallice,et al.  A failure of high level verbal response selection in progressive dynamic aphasia , 2005, Cognitive neuropsychology.

[28]  W. Sturm,et al.  Neuropsychological assessment , 2007, Journal of Neurology.

[29]  Tim Shallice,et al.  Case study approach in neuropsychological research , 1979 .

[30]  William H. Press,et al.  Numerical Recipes in Pascal , 2007 .

[31]  John R. Crawford,et al.  Psychometric Foundations of Neuropsychological Assessment , 2005 .

[32]  Max Coltheart,et al.  Cognitive Neuropsychology , 2014, Scholarpedia.

[33]  D. C. Howell,et al.  Comparing an Individual's Test Score Against Norms Derived from Small Samples , 1998 .

[34]  Paul H. Garthwaite,et al.  Wanted: Fully Operational Definitions of Dissociations in Single-Case Studies , 2003, Cortex.

[35]  F. Lopera,et al.  Brain potentials reflect residual face processing in a case of prosopagnosia , 2004, Cognitive neuropsychology.

[36]  M. Lezak.  Neuropsychological assessment, 3rd ed. , 1995 .

[37]  Lyndsey Nickels,et al.  Separating input and output phonology: semantic, phonological, and orthographic effects in short-term memory impairment , 2005, Cognitive neuropsychology.

[38]  P. Garthwaite,et al.  Evaluation of criteria for classical dissociations in single-case studies by Monte Carlo simulation. , 2005, Neuropsychology.

[39]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[40]  A. Benton,et al.  Journal of Clinical Neuropsychology , 1981 .

[41]  U. Frith,et al.  The impact of extensive medial frontal lobe damage on 'Theory of Mind' and cognition. , 2004, Brain : a journal of neurology.

[42]  Richard J. Brown.  Neuropsychology Mental Structure , 1989 .

[43]  Morris Moscovitch,et al.  Where to? Remote Memory for Spatial Relations and Landmark Identity in Former Taxi Drivers with Alzheimer's Disease and Encephalitis , 2005, Journal of Cognitive Neuroscience.

[44]  A. Azzalini,et al.  Distributions generated by perturbation of symmetry with emphasis on a multivariate skew t‐distribution , 2003, 0911.2342.

[45]  Jeremy MG Taylor,et al.  Robust Statistical Modeling Using the t Distribution , 1989 .

[46]  Michael McCloskey,et al.  Theory and evidence in cognitive neuropsychology: A "radical" response to Robertson, Knight, Rafal, and Shimamura (1993). , 1993 .