The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review.

In 2 meta-analyses involving 58 studies and 59,575 participants, we quantitatively summarized the relative reliability and validity of continuous (i.e., dimensional) and discrete (i.e., categorical) measures of psychopathology. Overall, results suggest an expected 15% increase in reliability and 37% increase in validity through adoption of a continuous over discrete measure of psychopathology alone. This increase occurs across all types of samples and forms of psychopathology, with little evidence for exceptions. For typical observed effect sizes, the increase in validity is sufficient to almost halve sample sizes necessary to achieve standard power levels. With important caveats, the current results, considered with previous research, provide sufficient empirical and theoretical basis to assume a priori that continuous measurement of psychopathology is more reliable and valid. Use of continuous measures in psychopathology assessment has widespread theoretical and practical benefits in research and clinical settings.

[1]  Janet B W Williams,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[2]  K. Markon How things fall apart: understanding the nature of internalizing through its relationship with impairment. , 2010, Journal of abnormal psychology.

[3]  C. Wilusz,et al.  Repeat expansion diseases: when a good RNA turns bad , 2010, Wiley interdisciplinary reviews. RNA.

[4]  S. Hyman,et al.  The diagnosis of mental disorders: the problem of reification. , 2010, Annual review of clinical psychology.

[5]  J. van os,et al.  Systematic reviews of categorical versus continuum models in psychosis: evidence for discontinuous subpopulations underlying a psychometric continuum. Implications for DSM-V, DSM-VI, and DSM-VII. , 2010, Annual review of clinical psychology.

[6]  J. Taylor,et al.  Repeat expansion disease: progress and puzzles in disease pathogenesis , 2010, Nature Reviews Genetics.

[7]  Marcello Gallucci,et al.  A conceptual and empirical examination of justifications for dichotomization. , 2009, Psychological methods.

[8]  J. H. Kamphuis,et al.  On categorical diagnoses in DSM-V: cutting dimensions at useful points? , 2009, Psychological assessment.

[9]  T. Widiger,et al.  Clinical utility and DSM-V. , 2009, Psychological assessment.

[10]  D. K. Marcus,et al.  Paranoid personality has a dimensional latent structure: taxometric analyses of community and clinical samples. , 2009, Journal of abnormal psychology.

[11]  K. Markon,et al.  Modeling psychopathology structure: a symptom-level analysis of Axis I and II disorders , 2009, Psychological Medicine.

[12]  A. Mitchell,et al.  Rate of progression of mild cognitive impairment to dementia – meta‐analysis of 41 robust inception cohort studies , 2009, Acta psychiatrica Scandinavica.

[13]  S. Haynes,et al.  Idiographic assessment: conceptual and psychometric foundations of individualized behavioral assessment. , 2009, Clinical psychology review.

[14]  Gitta Lubke,et al.  Distinguishing Between Latent Classes and Continuous Factors with Categorical Outcomes: Class Invariance of Parameters of Factor Mixture Models , 2008, Multivariate behavioral research.

[15]  S. Tiffany,et al.  On the nature of nicotine addiction: a taxometric analysis. , 2008, Journal of abnormal psychology.

[16]  J. Prisciandaro,et al.  A comparison of the predictive abilities of dimensional and categorical models of unipolar depression in the National Comorbidity Survey , 2008, Psychological Medicine.

[17]  O. Havik,et al.  Psychometric properties of Psychopathology checklists for Adults with Intellectual Disability (P-AID) on a community sample of adults with intellectual disability. , 2008, Research in developmental disabilities.

[18]  J. Dumas,et al.  Categorical and dimensional approaches to the measurement of disruptive behavior in the preschool years: a meta-analysis. , 2008, Clinical psychology review.

[19]  Laura S. Guy,et al.  Correspondence between self-report and interview-based assessments of antisocial personality disorder. , 2008, Psychological assessment.

[20]  H. Kraemer,et al.  Dimensional Approaches in Diagnostic Classification. Refining the Research Agenda for DSM-V , 2008 .

[21]  H. Kraemer,et al.  Comprar Dimensional Approaches in Diagnostic Classification. Refining the Research Agenda for DSM-V | Darrel A. Regier | 9780890423431 | American Psychiatric Publishing , 2008 .

[22]  J. Brooks Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[23]  T. Beauchaine,et al.  METHODOLOGICAL ARTICLE: A Brief Taxometrics Primer , 2007, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[24]  H. Merckelbach,et al.  The Dissociative Experiences Taxon Is Related to Fantasy Proneness , 2007, The Journal of nervous and mental disease.

[25]  N. Haslam The Latent Structure of Mental Disorders: A Taxometric Update on the Categorical vs Dimensional Debate , 2007 .

[26]  John Ruscio,et al.  Applying the Bootstrap to Taxometric Analysis: Generating Empirical Sampling Distributions to Help Interpret Results , 2007, Multivariate behavioral research.

[27]  D. Regier,et al.  Dimensional approaches to psychiatric classification: refining the research agenda for DSM‐V: an introduction , 2007, International journal of methods in psychiatric research.

[28]  J. Hunsley,et al.  Evidence-based assessment. , 2007, Annual review of clinical psychology.

[29]  E. Baca-García,et al.  Diagnostic stability of psychiatric disorders in clinical practice , 2007, British Journal of Psychiatry.

[30]  D. Rubin,et al.  Resolving the latent structure of schizophrenia endophenotypes using expectation-maximization-based finite mixture modeling. , 2007, Journal of abnormal psychology.

[31]  Kenneth N. Levy,et al.  The Personality Disorders Institute/Borderline Personality Disorder Research Foundation Randomized Control Trial for Borderline Personality Disorder: Reliability of Axis I and II Diagnoses , 2007, Psychiatric Quarterly.

[32]  Michael C Neale,et al.  Semi-Nonparametric Methods for Detecting Latent Non-normality: A Fusion of Latent Trait and Ordered Latent Class Modeling , 2006, Multivariate behavioral research.

[33]  M. Neale,et al.  Distinguishing Between Latent Classes and Continuous Factors: Resolution by Maximum Likelihood? , 2006, Multivariate behavioral research.

[34]  T. McGlashan,et al.  Comparison of alternative models for personality disorders , 2006, Psychological Medicine.

[35]  B. Grant,et al.  DSM-IV alcohol dependence: a categorical or dimensional phenotype? , 2006, Psychological Medicine.

[36]  R. Krueger,et al.  Information-theoretic latent distribution modeling: distinguishing discrete and continuous latent variable models. , 2006, Psychological methods.

[37]  S. Lilienfeld,et al.  Applications of Taxometric Methods to Problems of Comorbidity: Perspectives and Challenges , 2006 .

[38]  R. Krueger,et al.  Reinterpreting comorbidity: a model-based approach to understanding and classifying psychopathology. , 2006, Annual review of clinical psychology.

[39]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[40]  R. Krueger,et al.  Categorical and continuous models of liability to externalizing disorders: a direct comparison in NESARC. , 2005, Archives of general psychiatry.

[41]  Gregory T. Smith On construct validity: issues of method and measurement. , 2005, Psychological assessment.

[42]  C. Patrick,et al.  Externalizing psychopathology in adulthood: a dimensional-spectrum conceptualization and its implications for DSM-V. , 2005, Journal of abnormal psychology.

[43]  T. Widiger,et al.  Diagnostic categories or dimensions? A question for the Diagnostic And Statistical Manual Of Mental Disorders--fifth edition. , 2005, Journal of abnormal psychology.

[44]  Robert L Stout,et al.  Dimensional representations of DSM-IV personality disorders: relationships to functional impairment. , 2005, The American journal of psychiatry.

[45]  W. Horan,et al.  Examining the latent structure of negative symptoms: Is there a distinct subtype of negative symptom schizophrenia? , 2005, Schizophrenia Research.

[46]  J. Ioannidis Why Most Published Research Findings Are False , 2005 .

[47]  A. Beautrais,et al.  Subthreshold depression in adolescence and mental health outcomes in adulthood. , 2005, Archives of general psychiatry.

[48]  Timothy J Trull,et al.  Categorical and dimensional models of personality disorder. , 2005, Annual review of clinical psychology.

[49]  S. Guze,et al.  The neo-Kraepelinian revolution in psychiatric diagnosis , 2005, European Archives of Psychiatry and Clinical Neuroscience.

[50]  M. Meulders,et al.  A conceptual and psychometric framework for distinguishing categories and dimensions. , 2005, Psychological review.

[51]  Hok Pan Yuen,et al.  Two-year stability of personality disorder in older adolescent outpatients. , 2004, Journal of personality disorders.

[52]  H. D’haenen,et al.  The ADP-IV questionnaire: differential validity and concordance with the semi-structured interview. , 2004, Journal of personality disorders.

[53]  A. Seidler,et al.  Concordance of DSM-IV Axis I and II diagnoses by personal and informant's interview , 2004, Psychiatry Research.

[54]  Julian P T Higgins,et al.  Controlling the risk of spurious findings from meta‐regression , 2004, Statistics in medicine.

[55]  Nick Haslam,et al.  Categorical Versus Dimensional Models of Mental Disorder: The Taxometric Evidence , 2003, The Australian and New Zealand journal of psychiatry.

[56]  Arnoud Arntz,et al.  Short-interval test-retest interrater reliability of the Dutch version of the Structured Clinical Interview for DSM-IV personality disorders (SCID-II). , 2003, Journal of personality disorders.

[57]  T. McGlashan,et al.  The representation of four personality disorders by the schedule for nonadaptive and adaptive personality dimensional model of personality. , 2003, Psychological assessment.

[58]  A. Pickles,et al.  Natural categories or fundamental dimensions: On carving nature at the joints and the rearticulation of psychopathology , 2003, Development and Psychopathology.

[59]  B. Grant,et al.  The Alcohol Use Disorder and Associated Disabilities Interview Schedule-IV (AUDADIS-IV): reliability of alcohol consumption, tobacco use, family history of depression and psychiatric diagnostic modules in a general population sample. , 2003, Drug and alcohol dependence.

[60]  L. Benjamin,et al.  Validation of the Wisconsin Personality Disorders Inventory-IV with the SCID-II. , 2003, Journal of personality disorders.

[61]  A. Korten,et al.  Dimensional vs. categorical diagnosis in psychosis , 2003, Acta psychiatrica Scandinavica.

[62]  D. Watson Investigating the construct validity of the dissociative taxon: stability analyses of normal and pathological dissociation. , 2003, Journal of abnormal psychology.

[63]  Nick Haslam,et al.  The dimensional view of personality disorders: a review of the taxometric evidence. , 2003, Clinical psychology review.

[64]  R. Kendell,et al.  Distinguishing between the validity and utility of psychiatric diagnoses. , 2003, The American journal of psychiatry.

[65]  S. Shankman,et al.  Dimensional diagnosis of depression: adding the dimension of course to severity, and comparison to the DSM. , 2002, Comprehensive psychiatry.

[66]  F. Frankenburg,et al.  Inter-rater and test-retest reliability of the Revised Diagnostic Interview for Borderlines. , 2002, Journal of personality disorders.

[67]  D. Bonett Sample size requirements for estimating intraclass correlations with desired precision , 2002, Statistics in medicine.

[68]  J. Ruscio,et al.  A Structure-Based Approach to Psychological Assessment , 2002, Assessment.

[69]  V. Peralta,et al.  Classifying psychotic disorders: issues regarding categorial vs. dimensional approaches and time frame to assess symptoms , 2002, European Archives of Psychiatry and Clinical Neuroscience.

[70]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[71]  F. Frankenburg,et al.  Attainment and maintenance of reliability of axis I and II disorders over the course of a longitudinal study. , 2001, Comprehensive psychiatry.

[72]  A. Pickles,et al.  Child psychiatric symptoms and psychosocial impairment: relationship and prognostic significance , 2001, British Journal of Psychiatry.

[73]  L. Bierut,et al.  Towards a molecular epidemiology of alcohol dependence: analysing the interplay of genetic and environmental risk factors. , 2001, The British journal of psychiatry. Supplement.

[74]  K. Merikangas,et al.  Multi-dimensional criteria for the diagnosis of depression. , 2001, Journal of affective disorders.

[75]  T. Brown,et al.  Reliability of DSM-IV anxiety and mood disorders: implications for the classification of emotional disorders. , 2001, Journal of abnormal psychology.

[76]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[77]  C. Adams,et al.  Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophrenia , 2000, British Journal of Psychiatry.

[78]  J. Touchon,et al.  Mild cognitive impairment: conceptual basis and current nosological status , 2000, The Lancet.

[79]  P. Costa,et al.  The cross-cultural generalizability of Axis-II constructs: an evaluation of two personality disorder assessment instruments in the People's Republic of China. , 2000, Journal of personality disorders.

[80]  T. McGlashan,et al.  The Collaborative Longitudinal Personality Disorders Study: reliability of axis I and II diagnoses. , 2000, Journal of personality disorders.

[81]  R. Murray,et al.  A comparison of the utility of dimensional and categorical representations of psychosis , 1999, Psychological Medicine.

[82]  T. Treat,et al.  Quantifying the information value of clinical assessments with signal detection theory. , 1999, Annual review of psychology.

[83]  C. R. Cloninger,et al.  Personality and psychopathology , 1999 .

[84]  L. Clark Dimensional approaches to personality disorder assessment and diagnosis , 1999 .

[85]  K. Pötzelberger,et al.  The Asymptotic Loss of Information for Grouped Data , 1998 .

[86]  E. Lindström,et al.  DSM-IV and ICD-10 personality disorders: a comparison of a self-report questionnaire (DIP-Q) with a structured interview , 1998, European Psychiatry.

[87]  J. Schwartz,et al.  30-month stability of personality disorder diagnoses in depressed outpatients. , 1998, The American journal of psychiatry.

[88]  B. Mulsant,et al.  The relationship between patient, informant, social worker, and consensus diagnoses of personality disorder in elderly depressed inpatients. , 1998, The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry.

[89]  H. Tennen,et al.  Personality disorders in substance abusers: relation to substance use. , 1998, The Journal of nervous and mental disease.

[90]  P. Meehl,et al.  Multivariate taxometric procedures , 1998 .

[91]  Long-term test-retest reliability of personality disorder diagnoses in opiate dependent patients. , 1998, Journal of personality disorders.

[92]  A. Fossati,et al.  Brief communication: criterion validity of the Personality Diagnostic Questionnaire-4+ (PDQ-4+) in a mixed psychiatric sample. , 1998, Journal of personality disorders.

[93]  A. Arntz,et al.  Short-interval test-retest interrater reliability of the Structured Clinical Interview for DSM-III-R personality disorders (SCID-II) in outpatients. , 1998, Journal of personality disorders.

[94]  Paul E. Meehl,et al.  Multivariate Taxometric Procedures: Distinguishing Types from Continua , 1997 .

[95]  L. Krames,et al.  The continuity of depression in clinical and nonclinical samples. , 1997, Psychological bulletin.

[96]  N G Waller,et al.  The prevalence and biometric structure of pathological dissociation in the general population: taxometric and behavior genetic findings. , 1997, Journal of abnormal psychology.

[97]  A. Fossati,et al.  Interrater reliability and internal consistency of the structured clinical interview for DSM-IV axis II personality disorders (SCID-II), version 2.0. , 1997, Journal of personality disorders.

[98]  K. Wells,et al.  Socioeconomic burden of subsyndromal depressive symptoms and major depression in a sample of the general population. , 1996, The American journal of psychiatry.

[99]  P. Meehl,et al.  Taxometric Analysis: II. Detecting Taxonicity Using Covariance of Two Quantitative Indicators in Successive Intervals of a Third Indicator (Maxcov Procedure) , 1996 .

[100]  P. Sham,et al.  Psychopathological syndromes in the functional psychoses: associations with course and outcome , 1996, Psychological Medicine.

[101]  D. Fergusson,et al.  Predictive validity of categorically and dimensionally scored measures of disruptive childhood behaviors. , 1995, Journal of the American Academy of Child and Adolescent Psychiatry.

[102]  I. Gotlib,et al.  Symptoms versus a diagnosis of depression: differences in psychosocial functioning. , 1995, Journal of consulting and clinical psychology.

[103]  C. Sherbourne,et al.  Subthreshold depression and depressive disorder: clinical characteristics of general medical and mental health specialty outpatients. , 1994, The American journal of psychiatry.

[104]  S. Messick Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[105]  S. Messick Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[106]  Paul E. Meehl,et al.  Taxometric analysis. I: Detecting taxonicity with two quantitative indicators using means above and below a sliding cut (MAMBAC procedure) , 1994 .

[107]  S. Faraone,et al.  Measuring diagnostic accuracy in the absence of a "gold standard". , 1994, The American journal of psychiatry.

[108]  Y. Ono,et al.  The International Personality Disorder Examination. The World Health Organization/Alcohol, Drug Abuse, and Mental Health Administration international pilot study of personality disorders. , 1994, Archives of general psychiatry.

[109]  Thomas A. Widiger,et al.  Categorical Versus Dimensional Classification: Implications from and For Research , 1992 .

[110]  M. Weissman,et al.  Service utilization and social morbidity associated with depressive symptoms in the community. , 1992, JAMA.

[111]  P. Meehl Factors and Taxa, Traits and Types, Differences of Degree and Differences in Kind , 1992 .

[112]  A. Skodol,et al.  Diagnosis of DSM-III-R personality disorders by two structured interviews: patterns of comorbidity. , 1992, The American journal of psychiatry.

[113]  K. Sher,et al.  Temporal stability of the diagnostic interview schedule , 1991 .

[114]  H. Jackson,et al.  Concordance between two personality disorder instruments with psychiatric inpatients. , 1991, Comprehensive psychiatry.

[115]  Paul A. Pilkonis,et al.  Validity in the diagnosis of personality disorders : the use of the LEAD standard , 1991 .

[116]  L. George,et al.  Depression, disability days, and days lost from work in a prospective epidemiologic survey. , 1990, JAMA.

[117]  H. Skinner,et al.  Diagnostic validity of the MAST and the alcohol dependence scale in the assessment of DSM-III alcohol disorders. , 1990, Journal of studies on alcohol.

[118]  L. Siever,et al.  Structured interviews for borderline personality disorder. , 1990, The American journal of psychiatry.

[119]  W. Coryell,et al.  Diagnosing personality disorders in the community. A comparison of self-report and interview measures. , 1990, Archives of general psychiatry.

[120]  H. Jackson,et al.  Diagnosing Personality Disorders in Recent-Onset Schizophrenia , 1990, The Journal of nervous and mental disease.

[121]  H. Jackson,et al.  Personality Disorder Assessment for Psychiatric Inpatients , 1990, The Australian and New Zealand journal of psychiatry.

[122]  W. Coryell,et al.  The Reliability of Personality Disorder Diagnoses in a Nonpatient Sample , 1989 .

[123]  R. Spitzer,et al.  A comparison of clinical and self-report diagnoses of DSM-III personality disorders in 552 patients. , 1989, Comprehensive psychiatry.

[124]  H. Kraemer,et al.  2 x 2 kappa coefficients: measures of agreement or association. , 1989, Biometrics.

[125]  W. Coryell,et al.  Diagnosing personality disorder in depressed patients. A comparison of patient and informant interviews. , 1988, Archives of general psychiatry.

[126]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[127]  R. Tibshirani,et al.  Local Likelihood Estimation , 1987 .

[128]  M C Weinstein,et al.  Performance of screening and diagnostic tests. Application of receiver operating characteristic analysis. , 1987, Archives of general psychiatry.

[129]  T. Trull,et al.  A multidimensional scaling of the DSM-III personality disorders. , 1987, Archives of general psychiatry.

[130]  J. Oldham,et al.  The Personality Disorder Examination: A preliminary report. , 1987 .

[131]  Frank J. Landy,et al.  Stamp collecting versus science: Validation as hypothesis testing. , 1986 .

[132]  Mark Snyder,et al.  "To Carve Nature at Its Joints": On the Existence of Discrete Classes in Personality , 1985 .

[133]  R. Blashfield Feighner et al., invisible colleges, and the Matthew effect. , 1982, Schizophrenia bulletin.

[134]  Philip C. Kendall,et al.  Handbook of research methods in clinical psychology , 1982 .

[135]  R. Michels,et al.  Diagnostic and Statistical Manual of Mental Disorders, 3rd ed , 1981 .

[136]  H. Kraemer Ramifications of a population model forκ as a coefficient of reliability , 1979 .

[137]  P. Rothery A nonparametric measure of intraclass correlation , 1979 .

[138]  I. Gottesman,et al.  Reliability and validity in binary ratings: areas of common misunderstanding in diagnosis and symptom ratings. , 1978, Archives of general psychiatry.

[139]  A E Maxwell,et al.  Coefficients of Agreement Between Observers and Their Interpretation , 1977, British Journal of Psychiatry.

[140]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[141]  Further Consideration of the Distribution of the Multiple Correlation Coefficient , 1970 .

[142]  H. Eysenck The Classification of Depressive Illnesses , 1970, British Journal of Psychiatry.

[143]  R. Kendell The Classification of Depressive Illnesses , 1970, British Journal of Psychiatry.

[144]  S. Guze,et al.  Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia. , 1970, The American journal of psychiatry.

[145]  A Relatively Simple Form of the Distribution of the Multiple Correlation Coefficient , 1968 .

[146]  R. Dawes A note on base rates and psychometric efficiency. , 1962, Journal of consulting psychology.

[147]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[148]  P. Meehl,et al.  Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. , 1955, Psychological bulletin.

[149]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[150]  A. Lewis States of Depression* , 1938, British medical journal.