Diagnostic taxa as open concepts: Metatheoretical and statistical questions about reliability and construct validity in the grand strategy of nosological revision.

Since I find it hard to conceive that a rational mind could think otherwise, I presuppose that, ceteris paribus, careful delineation of the signs, symptoms, and course of a disorder (I cannot interest myself much in the semantic hassle over whether to call it “disease”) so as to increase the reliability of classifying clients or patients is desirable. While reliability and validity are not the same thing, it is a psychometric truism that the former bounds the latter, although it is worth mentioning that the bound is the square root of the reliability, so validity can theoretically be larger. Usually the operative validity (net attenuated construct validity) runs far below that upper bound set by the square root of the reliability coefficient. Hence, alterations in the format of assessment or in the content sampled, which might under some circumstances reduce reliability, could nevertheless increase the net attenuated construct validity. Similarly, changes in content or format that increase reliability may theoretically decrease validity. For instance, an alteration in the open-ended, unstructured format of Rorschach administration (as was attempted during World War II to make it possible to test large numbers of individuals and score reliably without inquiry) seemed to eliminate whatever slight validity the instrument had as usually administered. There is no mystery about this, although it is paradoxical at first look. We may be concerned about the reliability of behavior sampling by two different samplers (“interjudge agreement”) or with the trustworthiness of a sample as drawn by an individual judge (how many marbles do we draw from the urn, and how do we draw them?). In either case, the point is this: Whether an interview behavior or a psychological test item is viewed primar-

[1]  H. Reichenbach Experience And Prediction , 1938 .

[2]  A. H. Douthwaite Mistakes in diagnosis. , 1953, Medical world.

[3]  P. Meehl Hedonic capacity: some conjectures. , 1975, Bulletin of the Menninger Clinic.

[4]  Paul E. Meehl,et al.  Some ruminations on the validation of clinical procedures. , 1959 .

[5]  S. Spicker,et al.  Clinical Judgment: A Critical Appraisal , 1979 .

[6]  L. R. Goldberg Man versus model of man: just how conflicting is that evidence? , 1976 .

[7]  Symposium on Clinical and Statistical Prediction: The Tie That Binds. , 1956 .

[8]  Paul E. Meehl,et al.  Specific genetic etiology, psychodynamics, and therapeutic nihilism. , 1972 .

[9]  J. Sines Actuarial versus Clinical Prediction in Psychopathology , 1970, British Journal of Psychiatry.

[10]  P. Meehl,et al.  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. , 1955, Psychological bulletin.

[11]  Lewis R. Goldberg,et al.  Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences. , 1970 .

[12]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[13]  P. Meehl Why I Do Not Attend Case Conferences , 1973 .

[14]  E. Cureton Recipe for a cookbook. , 1957, Psychological bulletin.

[15]  S. Finn Base rates, utilities, and DSM-III: shortcomings of fixed-rule systems of psychodiagnosis. , 1982, Journal of abnormal psychology.

[16]  B Kleinmuntz,et al.  Computational and noncomputational clinical information processing by computer. , 1982, Behavioral science.

[17]  A. Caracciolo,et al.  Reduction-Sentences and Open Concepts , 1955 .

[18]  P. Meehl The cognitive activity of the clinician. , 1960 .

[19]  Paul E. Meehl,et al.  When shall we use our heads instead of the formula , 1957 .

[20]  M. S. Blois Clinical judgment and computers. , 1980, The New England journal of medicine.

[21]  E. Murphy,et al.  ONE CAUSE?MANY CAUSES?THE ARGUMENT FROM THE BIMODAL DISTRIBUTION. , 1964, Journal of chronic diseases.

[22]  Methods in Clinical Psychology, Vol. 1: Projective Assessment; Methods in Clinical Psychology, Vol. 2: Prediction and Research , 1979 .

[23]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[24]  P. Meehl,et al.  A funny thing happened to us on the way to the latent entities. , 1979, Journal of personality assessment.

[25]  T. Widiger Utilities and fixed diagnostic rules: comments on Finn. , 1983, Journal of abnormal psychology.

[26]  G. Āllport Personality: A Psychological Interpretation , 1938 .

[27]  Paul E. Meehl,et al.  Specific Etiology and Other Forms of Strong Influence: Some Quantitative Meanings , 1977 .

[28]  P. Meehl Manual for use with Checklist of Schizotypic Signs , 1973 .

[29]  D. W. Hands,et al.  Against Method: Outline of an Anarchistic Theory of Knowledge , 1977 .

[30]  P. Meehl Subjectivity in psychoanalytic inference: the nagging persistence of Wilhelm Fliess's Achensee question , 1983 .

[31]  H. Feigl,et al.  Minnesota studies in the philosophy of science , 1956 .

[32]  R. Cattell The Description and Measurement of Personality , 1947, Mental Health.

[33]  Laura Giat Roberto,et al.  Wanted: A good cookbook. , 1986 .

[34]  J. Earman Testing Scientific Theories , 1984 .

[35]  R. Holt Yet another look at clinical and statistical prediction: or, is clinical psychology worthwhile? , 1970, The American psychologist.

[36]  P. Hoch,et al.  Pseudoneurotic forms of schizophrenia , 1949, The Psychiatric quarterly.

[37]  L. G. Rorer,et al.  Optimum cutting scores to discriminate groups of unequal size and variance. , 1966, The Journal of applied psychology.

[38]  Philip C. Kendall,et al.  Handbook of research methods in clinical psychology , 1982 .

[39]  S. Finn Utility-balanced and utility-imbalanced rules: reply to Widiger. , 1983, Journal of abnormal psychology.

[40]  R. Dawes Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[41]  P. Meehl Schizotaxia, Schizotypy, Schizophrenia , 1962 .

[42]  J. Sawyer,et al.  Measurement and prediction, clinical and statistical. , 1966, Psychological bulletin.

[43]  Douthwaite Ah Mistakes in diagnosis. , 1953 .