Prediction of psychosis: model development and internal validation of a personalized risk calculator

BACKGROUND Over the past two decades, early detection and early intervention in psychosis have become essential goals of psychiatry. However, clinical impressions are insufficient for predicting psychosis outcomes in clinical high-risk (CHR) individuals; a more rigorous and objective model is needed. This study aims to develop and internally validate a model for predicting the transition to psychosis within 10 years. METHODS Two hundred and eight help-seeking individuals who fulfilled the CHR criteria were enrolled from the prospective, naturalistic cohort program for CHR at the Seoul Youth Clinic (SYC). The least absolute shrinkage and selection operator (LASSO)-penalized Cox regression was used to develop a predictive model for a psychotic transition. We performed k-means clustering and survival analysis to stratify the risk of psychosis. RESULTS The predictive model, which includes clinical and cognitive variables, identified the following six baseline variables as important predictors: 1-year percentage decrease in the Global Assessment of Functioning score, IQ, California Verbal Learning Test score, Strange Stories test score, and scores in two domains of the Social Functioning Scale. The predictive model showed a cross-validated Harrell's C-index of 0.78 and identified three subclusters with significantly different risk levels. CONCLUSIONS Overall, our predictive model showed a predictive ability and could facilitate a personalized therapeutic approach to different risks in high-risk individuals.

[1]  M. Hamilton.  The assessment of anxiety states by rating. , 1959, The British journal of medical psychology.

[2]  J. Os,et al.  The slow death of the concept of schizophrenia and the painful birth of the psychosis spectrum , 2017, Psychological Medicine.

[3]  A. Yung,et al.  Importance of Variable Selection in Multimodal Prediction Models in Patients at Clinical High Risk for Psychosis and Recent-Onset Depression. , 2019, JAMA psychiatry.

[4]  A. Yung,et al.  Declining transition rate in ultra high risk (prodromal) services: dilution or reduction of risk? , 2007, Schizophrenia bulletin.

[5]  C. Correll,et al.  Psychosis Prevention: A Modified Clinical High Risk Perspective From the Recognition and Prevention (RAP) Program. , 2015, The American journal of psychiatry.

[6]  Patrick D McGorry,et al.  Early intervention in psychosis: concepts, evidence and future directions , 2008, World psychiatry : official journal of the World Psychiatric Association.

[7]  M. Pruessner,et al.  The Clinic for Assessment of Youth at Risk (CAYR): 10 years of service delivery and research targeting the prevention of psychosis in Montreal, Canada , 2017, Early intervention in psychiatry.

[8]  R. Bentall,et al.  Abandoning the concept of 'schizophrenia': some implications of validity arguments for psychological research into psychotic phenomena. , 1988, The British journal of clinical psychology.

[9]  Trevor Hastie,et al.  Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent. , 2011, Journal of statistical software.

[10]  A. Riecher-Rössler,et al.  Prediction of transition to psychosis in patients with a clinical high risk for psychosis: a systematic review of methodology and reporting , 2017, Psychological Medicine.

[11]  P. McGuire,et al.  Improving Prognostic Accuracy in Subjects at Clinical High Risk for Psychosis: Systematic Review of Predictive Models and Meta-analytical Sequential Testing Simulation , 2016, Schizophrenia bulletin.

[12]  Y. Ahn,et al.  Validity and Reliability of the Korean Version of the Hamilton Depression Rating Scale(K-HDRS) , 2005 .

[13]  J. Addington,et al.  Intensive community outreach for those at ultra high risk of psychosis: dilution, not solution. , 2016, The lancet. Psychiatry.

[14]  T. McGlashan,et al.  The Dark Side of the Moon: Meta-analytical Impact of Recruitment Strategies on Risk Enrichment in the Clinical High Risk State for Psychosis Paolo Fusar-Poli*,1–3, Frauke Schultze-Lutter4, Marco Cappucciati1,3, Grazia Rutigliano1, Ilaria Bonoldi1–3, Daniel Stahl5, Stephan Borgwardt6, Anita Riecher-R , 2015 .

[15]  이정희,et al.  Controlled Oral Word Association Test , 2000, Definitions.

[16]  Jimmy Lee,et al.  Longitudinal Youth‐At‐Risk Study (LYRIKS): outreach strategies based on a community‐engaged framework , 2014, Early intervention in psychiatry.

[17]  P. Fusar-Poli,et al.  Short clinically-based prediction model to forecast transition to psychosis in individuals at clinical high risk state , 2019, European Psychiatry.

[18]  Predicting the onset of psychosis in patients at clinical high risk: practical guide to probabilistic prognostic reasoning , 2016, Evidence-Based Mental Health.

[19]  Holly Janes,et al.  Assessing the Value of Risk Predictions by Using Risk Stratification Tables , 2008, Annals of Internal Medicine.

[20]  P. McGuire,et al.  Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. , 2012, Archives of general psychiatry.

[21]  A. Riecher-Rössler,et al.  Development and Validation of a Dynamic Risk Prediction Model to Forecast Psychosis Onset in Patients at Clinical High Risk. , 2019, Schizophrenia bulletin.

[22]  S. Borgwardt,et al.  Moving beyond transition outcomes: Meta-analysis of remission rates in individuals at high clinical risk for psychosis , 2013, Psychiatry Research.

[23]  Tae Young Lee,et al.  Early intervention in psychosis: Insights from Korea. , 2012, Asian journal of psychiatry.

[24]  T. McGlashan,et al.  At risk or not at risk? A meta‐analysis of the prognostic accuracy of psychometric interviews for psychosis prediction , 2015, World psychiatry : official journal of the World Psychiatric Association.

[25]  Tyrone D. Cannon,et al.  Premorbid functional development and conversion to psychosis in clinical high-risk youths , 2013, Development and Psychopathology.

[26]  Tyrone D. Cannon,et al.  Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. , 2008, Archives of general psychiatry.

[27]  G. Juckel,et al.  Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European prediction of psychosis study. , 2010, Archives of general psychiatry.

[28]  R. McCarley,et al.  Prediction of psychosis in prodrome: development and validation of a simple, personalized risk calculator , 2018, Psychological Medicine.

[29]  E F Cook,et al.  Prediction of the need for intensive care in patients who come to emergency departments with acute chest pain. , 1996, The New England journal of medicine.

[30]  S. Wood,et al.  Transition to first episode psychosis in ultra high risk populations: Does baseline functioning hold the key? , 2013, Schizophrenia Research.

[31]  S. Woods,et al.  The Role of Cognition and Social Functioning as Predictors in the Transition to Psychosis for Youth With Attenuated Psychotic Symptoms , 2017, Schizophrenia bulletin.

[32]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[33]  J. Y. Park,et al.  Attribution bias in ultra-high risk for psychosis and first-episode schizophrenia , 2010, Schizophrenia Research.

[34]  P. McGuire,et al.  Transdiagnostic Risk Calculator for the Automatic Detection of Individuals at Risk and the Prediction of Psychosis: Second Replication in an Independent National Health Service Trust , 2018, Schizophrenia bulletin.

[35]  F. Crescenzo,et al.  Twelve-month psychosis-predictive value of the ultra-high risk criteria in children and adolescents , 2015, Schizophrenia Research.

[36]  G. Chelune,et al.  Developmental norms for the Wisconsin Card Sorting test. , 1986, Journal of clinical and experimental neuropsychology.

[37]  Tyrone D. Cannon,et al.  Specificity of Incident Diagnostic Outcomes in Patients at Clinical High Risk for Psychosis. , 2015, Schizophrenia bulletin.

[38]  N. Koutsouleris,et al.  Importance of Variable Selection in Multimodal Prediction Models in Patients at Clinical High Risk for Psychosis and Recent Onset Depression-Reply. , 2019, JAMA psychiatry.

[39]  M. Birchwood,et al.  Early intervention in psychosis: the critical-period hypothesis , 1998 .

[40]  A. Aleman,et al.  Individualized Prediction of Transition to Psychosis in 1,676 Individuals at Clinical High Risk: Development and Validation of a Multivariable Prediction Model Based on Individual Patient Data Meta-Analysis , 2019, Front. Psychiatry.

[41]  S. Ruhrmann,et al.  ‘A Rose Is a Rose Is a Rose’, but At-Risk Criteria Differ , 2012, Psychopathology.

[42]  J. Kwon,et al.  Pharmacotherapy and clinical characteristics of ultra‐high‐risk for psychosis according to conversion status: a naturalistic observational study , 2012, Early intervention in psychiatry.

[43]  Tyrone D. Cannon,et al.  At clinical high risk for psychosis: outcome for nonconverters. , 2011, The American journal of psychiatry.

[44]  P. Cuijpers,et al.  Cognitive behavioral therapy for subjects at ultrahigh risk for developing psychosis: a randomized controlled clinical trial. , 2012, Schizophrenia bulletin.

[45]  H. van Engeland,et al.  Neurocognitive and Clinical Predictors of Long-Term Outcome in Adolescents at Ultra-High Risk for Psychosis: A 6-Year Follow-Up , 2014, PloS one.

[46]  R. Reitan.  Validity of the Trail Making Test as an Indicator of Organic Brain Damage , 1958 .

[47]  Tae Young Lee,et al.  Symptomatic and functional remission of subjects at clinical high risk for psychosis: A 2-year naturalistic observational study , 2014, Schizophrenia Research.

[48]  K. Juston Osborne,et al.  External validation and extension of the NAPLS-2 and SIPS-RC personalized risk calculators in an independent clinical high-risk sample , 2019, Psychiatry Research.

[49]  T. Kim,et al.  A Normative Study of Lexical Verbal Fluency in an Educationally-Diverse Elderly Population , 2013, Psychiatry investigation.

[50]  Ewout W. Steyerberg,et al.  The Science of Prognosis in Psychiatry: A Review , 2018, JAMA psychiatry.

[51]  R. Tibshirani.  The lasso method for variable selection in the Cox model. , 1997, Statistics in medicine.

[52]  U. Gschwandtner,et al.  The Basel early‐detection‐of‐psychosis (FEPSY)‐study – design and preliminary results , 2007, Acta psychiatrica Scandinavica.

[53]  A. Nierenberg,et al.  Predictive analytics in mental health: applications, guidelines, challenges and perspectives , 2017, Molecular Psychiatry.

[54]  P. McGuire,et al.  Deconstructing Pretest Risk Enrichment to Optimize Prediction of Psychosis in Individuals at Clinical High Risk. , 2016, JAMA psychiatry.

[55]  P. McGuire,et al.  Speed of Psychosis Progression in People at Ultra-High Clinical Risk: A Complementary Meta-analysis. , 2015, JAMA psychiatry.

[56]  M. Nordentoft,et al.  Predictors of remission from the ultra‐high risk state for psychosis , 2020, Early intervention in psychiatry.

[57]  Paolo Fusar-Poli,et al.  Development and Validation of a Clinically Based Risk Calculator for the Transdiagnostic Prediction of Psychosis , 2017, JAMA psychiatry.

[58]  C. Carter,et al.  Personalized Prediction of Psychosis: External Validation of the NAPLS-2 Psychosis Risk Calculator With the EDIPPP Project. , 2016, The American journal of psychiatry.

[59]  P. McGuire,et al.  Transdiagnostic Individualized Clinically Based Risk Calculator for the Detection of Individuals at Risk and the Prediction of Psychosis: Model Refinement Including Nonlinear Effects of Age , 2019, Front. Psychiatry.

[60]  A. Riecher-Rössler,et al.  Correlations between self‐rating and observer‐rating of psychopathology in at‐risk mental state and first‐episode psychosis patients: influence of disease stage and gender , 2017, Early intervention in psychiatry.

[61]  Frank Davidoff,et al.  Predicting Clinical States in Individual Patients , 1996, Annals of Internal Medicine.

[62]  E. Steyerberg.  Clinical Prediction Models , 2008, Statistics for Biology and Health.

[63]  Tae Young Lee,et al.  Heterogeneity of Psychosis Risk Within Individuals at Clinical High Risk: A Meta-analytical Stratification. , 2016, JAMA psychiatry.

[64]  J K Kim,et al.  Normative study of the Korean-California Verbal Learning Test (K-CVLT). , 1999, The Clinical neuropsychologist.

[65]  Gregory Y H Lip,et al.  Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the euro heart survey on atrial fibrillation. , 2010, Chest.

[66]  J. Lieberman,et al.  A predictive model for conversion to psychosis in clinical high-risk patients , 2018, Psychological Medicine.

[67]  Hok Pan Yuen,et al.  Long-term follow-up of a group at ultra high risk ("prodromal") for psychosis: the PACE 400 study. , 2013, JAMA psychiatry.

[68]  Jun Soo Kwon,et al.  Clinical and empirical applications of the Rey–Osterrieth Complex Figure Test , 2006, Nature Protocols.

[69]  H. Sox,et al.  Clinical prediction rules. Applications and methodological standards. , 1985, The New England journal of medicine.

[70]  Tyrone D. Cannon,et al.  Lack of Diagnostic Pluripotentiality in Patients at Clinical High Risk for Psychosis: Specificity of Comorbidity Persistence and Search for Pluripotential Subgroups , 2018, Schizophrenia bulletin.

[71]  R. Bentall,et al.  Three-year follow-up of a randomized controlled trial of cognitive therapy for the prevention of psychosis in people at ultrahigh risk. , 2007, Schizophrenia bulletin.

[72]  J. Addington,et al.  Predictors of Transition to Psychosis in Individuals at Clinical High Risk , 2019, Current Psychiatry Reports.

[73]  Tyrone D. Cannon,et al.  An Individualized Risk Calculator for Research in Prodromal Psychosis. , 2016, The American journal of psychiatry.

[74]  R. Emsley,et al.  Cognitive Mechanisms of Change in Delusions: An Experimental Investigation Targeting Reasoning to Effect Change in Paranoia , 2014, Schizophrenia bulletin.

[75]  Stephan Ruhrmann,et al.  Improving the clinical prediction of psychosis by combining ultra-high risk criteria and cognitive basic symptoms , 2014, Schizophrenia Research.

[76]  G C Patton,et al.  Monitoring and care of young people at incipient risk of psychosis. , 1996, Schizophrenia bulletin.

[77]  Kazunori Matsumoto,et al.  A naturalistic longitudinal study of at-risk mental state with a 2.4year follow-up at a specialized clinic setting in Japan , 2014, Schizophrenia Research.

[78]  A. Yung,et al.  Can Clinicians Predict Psychosis in an Ultra High Risk Group? , 2010, The Australian and New Zealand journal of psychiatry.

[79]  S. Wood,et al.  Outcomes of nontransitioned cases in a sample at ultra-high risk for psychosis. , 2015, The American journal of psychiatry.

[80]  L. Bour,et al.  Psychosis prediction: stratification of risk estimation with information-processing and premorbid functioning variables. , 2014, Schizophrenia bulletin.

[81]  Lu Liu,et al.  A randomized controlled trial of cognitive behavioral therapy for individuals at clinical high risk of psychosis , 2011, Schizophrenia Research.

[82]  H. Yamasue,et al.  A multimodal approach to investigate biomarkers for psychosis in a clinical setting: The integrative neuroimaging studies in schizophrenia targeting for early intervention and prevention (IN-STEP) project , 2013, Schizophrenia Research.

[83]  Tae Young Lee,et al.  Neurocognitive function as a possible marker for remission from clinical high risk for psychosis , 2014, Schizophrenia Research.

[84]  J. Kwon,et al.  OPEN ACCESS The Reliability and Validity of the Korean Version of the Structured Interview for Prodromal Syndrome , 2022 .

[85]  P. McGuire,et al.  What Causes the Onset of Psychosis in Individuals at Clinical High Risk? A Meta-analysis of Risk and Protective Factors , 2019, Schizophrenia bulletin.

[86]  Heejung Bang,et al.  How to Establish Clinical Prediction Models , 2016, Endocrinology and metabolism.

[87]  J. Klosterkötter,et al.  A stratified model for psychosis prediction in clinical practice. , 2014, Schizophrenia bulletin.

[88]  J. O. Johannessen,et al.  Early detection and intervention with schizophrenia: rationale. , 1996, Schizophrenia bulletin.

[89]  R. McCarley,et al.  S105. VALIDATING THE PREDICTIVE ACCURACY OF THE NAPLS-2 PSYCHOSIS RISK CALCULATOR IN A CLINICAL HIGH-RISK SAMPLE FROM THE SHARP (SHANGHAI AT RISK FOR PSYCHOSIS) PROGRAM , 2018, The American journal of psychiatry.

[90]  A. Mackinnon,et al.  Dynamic prediction of transition to psychosis using joint modelling , 2018, Schizophrenia Research.

[91]  T. McGlashan,et al.  The psychosis high-risk state: a comprehensive state-of-the-art review. , 2013, JAMA psychiatry.