Parallel but not equivalent: Challenges and solutions for repeated assessment of cognition over time

Objective. Analyses of individual differences in change may be unintentionally biased when versions of a neuropsychological test used at different follow-ups are not of equivalent difficulty. This study's objective was to compare mean, linear, and equipercentile equating methods and demonstrate their utility in longitudinal research. Study design and setting: The Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE, N = 1,401) study is a longitudinal randomized trial of cognitive training. The Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 819) is an observational cohort study. Nonequivalent alternate versions of the Auditory Verbal Learning Test (AVLT) were administered in both studies. Results. Using visual displays, raw and mean-equated AVLT scores in both studies showed obvious nonlinear trajectories in reference groups that should show minimal change and poor equivalence over time (ps ≤ .001), and raw scores demonstrated poor fits in models of within-person change (root mean square errors of approximation, RMSEAs > 0.12). Linear and equipercentile equating produced more similar means in reference groups (ps ≥ .09) and performed better in growth models (RMSEAs < 0.05). Conclusion. Equipercentile equating is the preferred equating method because it accommodates tests more difficult than a reference test at different percentiles of performance and performs well in models of within-person trajectory. The method has broad applications in both clinical and research settings to enhance the ability to use nonequivalent test forms.

[1]  M. Albert,et al.  Cerebrospinal fluid profiles and prospective course and outcome in patients with amnestic mild cognitive impairment. , 2011, Archives of neurology.

[2]  Comparison of list B and list C of the rey auditory verbal learning test , 1997 .

[3]  T. Salthouse,et al.  Multivariate modeling of age and retest in longitudinal studies of cognitive abilities. , 2005, Psychology and aging.

[4]  Philip S. Insel,et al.  Development and assessment of a composite score for memory in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) , 2012, Brain Imaging and Behavior.

[5]  Stefan Leucht,et al.  What does the PANSS mean? , 2005, Schizophrenia Research.

[6]  A. Gross,et al.  Memory training and strategy use in older adults: results from the ACTIVE study. , 2011, Psychology and aging.

[7]  Hermann Ebbinghaus (1885).  Memory: A Contribution to Experimental Psychology , 2013, Annals of Neurosciences.

[8]  K. Boone,et al.  Handbook of Normative Data for Neuropsychological Assessment , 1999 .

[9]  S. C. Bates,et al.  Methods in Behavioral Research, 11th Edition , 2011 .

[10]  Emilio Ferrer,et al.  Modeling age and retest processes in longitudinal studies of cognitive abilities. , 2004, Psychology and aging.

[11]  H. Möller,et al.  Does Clinical Judgment of Baseline Severity and Changes in Psychopathology Depend on the Patient Population?: Results of a CGI and PANSS Linking Analysis in a Naturalistic Study , 2010, Journal of clinical psychopharmacology.

[12]  Hélène Jacqmin-Gadda,et al.  A Nonlinear Model with Latent Process for Cognitive Evolution Using Multivariate Longitudinal Data , 2006, Biometrics.

[13]  K. Langa,et al.  Development and validation of a brief cognitive assessment tool: the sweet 16. , 2011, Archives of internal medicine.

[14]  T. Salthouse,et al.  Implications of short-term retest effects for the interpretation of longitudinal change. , 2008, Neuropsychology.

[15]  K. Ball,et al.  Long-term effects of cognitive training on everyday functional outcomes in older adults. , 2006, JAMA.

[16]  S. Woods,et al.  Evidence‐based guidelines for interpretation of the Panic Disorder Severity Scale , 2009, Depression and anxiety.

[17]  L. Light.  Memory and aging: four hypotheses in search of data. , 1991, Annual review of psychology.

[18]  岩坪 威,et al.  Alzheimer's Disease Neuroimaging Initiative (ADNI)の最新情報 (特集 アルツハイマー病の根本治療を目指す最近の進歩) , 2012 .

[19]  J. Brandt,et al.  Word list memory predicts everyday function and problem-solving in the elderly: Results from the ACTIVE cognitive intervention trial , 2011, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[20]  John R. Anderson,et al.  RECOGNITION AND RETRIEVAL PROCESSES IN FREE RECALL , 1972 .

[21]  W. Montague,et al.  Category norms of verbal items in 56 categories A replication and extension of the Connecticut category norms , 1969 .

[22]  P. Rubé,et al.  L’examen Clinique en Psychologie , 1959 .

[23]  T. Stijnen,et al.  Review: a gentle introduction to imputation of missing values. , 2006, Journal of clinical epidemiology.

[24]  D. Borsboom.  Measuring the mind: Conceptual issues in contemporary psychometrics , 2005 .

[25]  L. San,et al.  Validation of the Excited Component of the Positive and Negative Syndrome Scale (PANSS-EC) in a naturalistic sample of 278 patients with acute psychosis and agitation in a psychiatric emergency room , 2011, Health and quality of life outcomes.

[26]  D. Amtmann,et al.  Measuring fatigue in persons with multiple sclerosis: creating a crosswalk between the Modified Fatigue Impact Scale and the PROMIS Fatigue Short Form , 2012, Quality of Life Research.

[27]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[28]  G. Geffen,et al.  Test-retest reliability of a new form of the auditory verbal learning test (AVLT). , 1994, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[29]  W. Gouvier,et al.  A comparison of list B and list C of the Rey auditory verbal learning test , 1992 .

[30]  Z. Khachaturian.  Alzheimer's & Dementia: The Journal of the Alzheimer's Association , 2008, Alzheimer's & Dementia.

[31]  Emilio Ferrer,et al.  Estimating retest effects in longitudinal assessments of cognitive functioning in adults between 18 and 60 years of age. , 2004, Developmental psychology.

[32]  R. Brennan,et al.  Test equating : methods and practices , 1995 .

[33]  Vikas Singh,et al.  Predictive markers for AD in a multi-modality framework: An analysis of MCI progression in the ADNI population , 2011, NeuroImage.

[34]  George W Rebok,et al.  Cognitive predictors of everyday functioning in older adults: results from the ACTIVE Cognitive Intervention Trial. , 2011, The journals of gerontology. Series B, Psychological sciences and social sciences.

[35]  J. Fisher,et al.  Neuropsychological Assessment, 2nd Ed , 1985 .

[36]  P. Cozby,et al.  Methods in behavioral research, 5th ed. , 1993 .

[37]  A. McNeil,et al.  Latent Curve Models: A Structural Equation Approach , 2007 .

[38]  Benton J. Underwood,et al.  Coding processes in verbal learning , 1963 .

[39]  C. Jack,et al.  Alzheimer's Disease Neuroimaging Initiative , 2008 .

[40]  Danielle J. Harvey,et al.  The Alzheimer's Disease Neuroimaging Initiative: Annual change in biomarkers and clinical outcomes , 2010, Alzheimer's & Dementia.

[41]  James F. Malec,et al.  The Auditory-Verbal Learning Test (AVLT): Norms for ages 55 years and older. , 1990 .

[42]  K. Boone,et al.  Handbook of normative data for neuropsychological assessment, 2nd ed. , 2005 .

[43]  J R Crawford,et al.  Demonstration of savings on the AVLT and development of a parallel form. , 1989, Journal of clinical and experimental neuropsychology.

[44]  George W Rebok,et al.  Effects of cognitive training interventions with older adults: a randomized controlled trial. , 2002, JAMA.

[45]  J N Morris,et al.  ACTIVE: a cognitive intervention trial to promote independence in older adults. , 2001, Controlled clinical trials.

[46]  T. Salthouse.  Influence of age on practice effects in longitudinal neurocognitive change. , 2010, Neuropsychology.

[47]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[48]  E. Marcantonio,et al.  Telephone Interview for Cognitive Status: Creating a crosswalk with the Mini-Mental State Examination , 2009, Alzheimer's & Dementia.

[49]  G. Pearlson,et al.  Alternative Forms of the Rey Auditory Verbal Learning Test: A Review , 2005, Behavioural neurology.

[50]  Herman Buschke,et al.  Selective reminding for analysis of memory and learning , 1973 .

[51]  Bengt Muthén,et al.  General Longitudinal Modeling of Individual Differences in Experimental Designs: A Latent Variable Framework for Analysis and Power Estimation , 1997 .

[52]  Owen Carmichael,et al.  Longitudinal changes in white matter disease and cognition in the first year of the Alzheimer disease neuroimaging initiative. , 2010, Archives of neurology.

[53]  Anders M. Dale,et al.  Six-month atrophy in MTL structures is associated with subsequent memory decline in elderly controls , 2010, NeuroImage.

[54]  Jeanine M. Parisi,et al.  Modeling change in memory performance and memory perceptions: findings from the ACTIVE study. , 2011, Psychology and aging.

[55]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[56]  P. Cozby,et al.  Methods in behavioral research , 1977 .

[57]  P. Diggle,et al.  Practice and drop-out effects during a 17-year longitudinal study of cognitive aging. , 2004, The journals of gerontology. Series B, Psychological sciences and social sciences.

[58]  Allan Paivio,et al.  A factor-analytic study of word attributes and verbal learning. , 1968 .