Cognitive profile analysis: A shared professional myth.

Interpretation of cognitive test profiles is popular in school psychological training and practice but is not supported by research evidence. This special issue contains four studies that provide additional data regarding cognitive test profiles. Consistent with the largely negative research literature, detailed analysis found all four cognitive profile reports lack­ ing reliability, validity, or diagnostic utility; even cognitive profiles composed of com­ posites were psychometrically weak. These results were not surprising because ipsative methods are inferior to normative methods in cognitive assessment. Given this consistent failure of empirical validation, belief in the utility of cognitive test profile interpretations was likened to a shared professional myth and it was recommended that psychologists es­ chew the application of cognitive test profiles for differential diagnosis and remediation.

[1]  J. Hunter.  Cognitive ability, cognitive aptitudes, job knowledge, and job performance , 1986 .

[2]  J. Glutting,et al.  Informing Stylistic Learning Behavior, Disposition, and Achievement through Ability Subtests--Or, More Illusions of Meaning?. , 1997 .

[3]  E. Youngstrom,et al.  Incremental efficacy of WISC–III factor scores in predicting achievement: What do they tell us? , 1997 .

[4]  A. Jensen,et al.  Commentary: Vehicles of g , 1992 .

[5]  N. Todorov.  Intelligent Testing with the WISC-III,Kaufman Alan. S.,NY:John Wiley & Sons (1994),401 pp., $79.95 hardcover , 1995, Behaviour Change.

[6]  R L Stubblefield,et al.  Behavioral sciences and the law. , 1966, The American journal of orthopsychiatry.

[7]  N. Meyers,et al.  H = W. , 1964, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Marley W. Watkins,et al.  More ups and downs of subtest analysis: Criterion validity of the DAS with an unselected cohort. , 1998 .

[9]  Alexander von Eye,et al.  Configural Frequency Analysis in Applied Psychological Research , 1996 .

[10]  T. Oakland,et al.  The Course on Individual Cognitive Assessment , 2000 .

[11]  R. Dawes,et al.  Heuristics and Biases: Clinical versus Actuarial Judgment , 2002 .

[12]  Harvey A. Skinner,et al.  Differentiating the Contribution of Elevation, Scatter and Shape in Profile Similarity , 1978 .

[13]  Relative usefulness of elevation, variability, and shape information from WISC—R, K-ABC, and Fourth Edition Stanford-Binet profiles in predicting achievement. , 1992 .

[14]  H. E. Tinsley,et al.  Uses of factor analysis in counseling psychology research. , 1987 .

[15]  J. Naglieri.  Can profile analysis of ability test scores work? An illustration using the PASS theory and CAS with an unselected cohort. , 2000 .

[16]  Jack A. Naglieri,et al.  Cognitive Assessment System. , 1997 .

[17]  K. Bailey.  Typologies and taxonomies: An introduction to classification techniques. , 1994 .

[18]  David Faust,et al.  Data integration in legal evaluations: Can clinicians deliver on their premises? , 1989 .

[19]  M. Zimmerman,et al.  Reporting Errors in Studies of the Diagnostic Performance of Self-Administered Questionnaires: Extent of the Problem, Recommendations for Standardized Presentation of Results, and Implications for the Peer Review Process. , 1993 .

[20]  Marley W. Watkins,et al.  Wechsler Subtest Analysis: The Right Way, The Wrong Way, or No Way?. , 1994 .

[21]  G. Hynd,et al.  Measurable Biological Substrates to Verbal-Performance Differences in Wechsler Scores. , 2000 .

[22]  K. Stanovich.  The Sociopsychometrics of Learning Disabilities , 1999, Journal of learning disabilities.

[23]  P. Meehl,et al.  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. , 1955, Psychological bulletin.

[24]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[25]  A. Jensen,et al.  The g factor , 1996, Nature.

[26]  R. Short.  A Meta-Exploration of WISC-R Factor Score Profiles as a Function of Diagnosis And Intellectual Level. , 1986 .

[27]  M. Watkins.  Diagnostic Utility of the WISC-III Developmental Index as a Predictor of Learning Disabilities , 1996, Journal of learning disabilities.

[28]  C. R. Reynolds.  Putting the Individual into Aptitude-Treatment Interaction , 1988, Exceptional children.

[29]  Harvey A. Skinner,et al.  Modal Profile Analysis: a Computer Program for Classification Research , 1980 .

[30]  Joshua D. Lipsitz,et al.  Wechsler Comprehension and Picture Arrangement Subtests and Social Adjustment. , 1993 .

[31]  P. Meehl.  Credentialed Persons, Credentialed Knowledge , 1997 .

[32]  L. Reddy,et al.  The Practitioner's View of IQ Testing and Profile Analysis , 2000 .

[33]  Scientist-Practitioner Perspectives on Test Interpretation , 1998 .

[34]  D. Cicchetti.  Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. , 1994 .

[35]  A. Eye,et al.  Lead Article Configural Frequency Analysis in Applied Psychological Research , 1996 .

[36]  David Faust,et al.  School Psychologists' Diagnostic Decision-Making Processes: Objective-Subjective Discrepancies. , 1998 .

[37]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[38]  D. Zarin,et al.  Diagnostic decision making in psychiatry. , 1993, The American journal of psychiatry.

[39]  Marley W. Watkins,et al.  Discriminant and predictive validity of the WISC-III ACID profile among children with learning disabilities , 1997 .

[40]  A. Kaufman.  Intelligent testing with the WISC-R , 1979 .

[41]  John O. Willis,et al.  Intrasubtest Scatter on the WISC-Ill For Various Clinical Samples vs. The Standardization Sample: An Examination of WISC Folklore , 1995 .

[42]  David Faust,et al.  Research on human judgment and its application to clinical practice. , 1986 .

[43]  Kenneth A. Kavale,et al.  A Meta-Analysis of the Validity of Wechsler Scale Profiles and Recategorizations: Patterns Or Parodies? , 1984 .

[44]  R. Hale,et al.  Profile Analysis of the Wechsler Intelligence Scale for Children-Revised , 1983 .

[45]  M. Watkins,et al.  Prevalence and diagnostic utility of the WISC–III SCAD profile among children with disabilities. , 1997 .

[46]  J. Kramer,et al.  The Viability of Scatter Analysis On the Wisc-r aNd the Sbis: Examining a Vestige , 1987 .

[47]  Cecil R. Reynolds,et al.  Modal Profiles for the WISC-III. , 2000 .

[48]  R. Elwood.  Psychological tests and clinical discriminations: Beginning to address the base rate problem , 1993 .

[49]  I. Weiner.  On competence and ethicality in psychodiagnostic assessment. , 1989, Journal of personality assessment.

[50]  Morris J. Cohen,et al.  The Third and Fourth Factors of the WISC-III: What they Don't Measure , 1997 .

[51]  M. Watkins.  Diagnostic Utility of WISC-III Subtest Variability among Students with Learning Disabilities , 1999 .

[52]  M. Watkins,et al.  Incremental validity of WISC-III profile elevation, scatter, and shape information for predicting reading and math achievement , 2000 .

[53]  D. Beebe,et al.  Evaluation of the validity of the Wechsler Intelligence Scale for Children--Third Edition comprehension and picture arrangement subtests as measures of social intelligence. , 2000, Psychological assessment.

[54]  P. Meehl,et al.  Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. , 1996 .

[55]  E. Levinson,et al.  Heuristic principles and cognitive bias in decision making: Implications for assessment in school psychology. , 1993 .

[56]  E. Youngstrom,et al.  Incremental Efficacy of Differential Ability Scales Factor Scores in Predicting Individual Achievement Criteria , 1999 .

[57]  J. Fantuzzo,et al.  Just Say No to Subtest Analysis: A Critique on Wechsler Theory and Practice , 1990 .

[58]  R. Sternberg,et al.  Intelligence: Knowns and unknowns. , 1996 .

[59]  W. Velicer,et al.  Comparison of five rules for determining the number of components to retain. , 1986 .

[60]  C. Reynolds,et al.  Configural frequency analysis as a method of determining Wechsler profile types. , 2000 .

[61]  P. A. Teeter,et al.  Assessment of emotionally disturbed children with the WISC—III. , 1998 .

[62]  Bernhard Wilpert,et al.  Applied psychology : an international review , 1989 .

[63]  E. Somoza,et al.  Neuropsychiatric decision making: the role of disorder prevalence in diagnostic testing. , 1991, The Journal of neuropsychiatry and clinical neurosciences.