Developing and Validating an Individualized Clinical Prediction Model to Forecast Psychotic Recurrence in Acute and Transient Psychotic Disorders: Electronic Health Record Cohort Study.

Acute and transient psychotic disorders (ATPDs) include short-lived psychotic episodes with a high probability of developing psychotic recurrences. Clinical care for ATPD is currently limited by the inability to predict outcomes. Real-world electronic health record (EHR)-based retrospective cohort study STROBE/RECORD compliant included all individuals accessing the South London and Maudsley NHS Trust between 2006 and 2017 and receiving a first diagnosis of ATPD (F23, ICD-10). After imputing missing data, stepwise and LASSO Cox regression methods employing a priori predictors (n = 23) were compared to develop and internally validate an individualized risk prediction model to forecast the risk of psychotic recurrences following TRIPOD guidelines. The primary outcome was prognostic accuracy (area under the curve [AUC]). 3018 ATPD individuals were included (average age = 33.75 years, 52.7% females). Over follow-up (average 1042 ± 1011 days, up to 8 years) there were 1160 psychotic recurrences (events). Stepwise (n = 12 predictors) and LASSO (n = 17 predictors) regression methods yielded comparable prognostic accuracy, with an events per variable ratio >100 for both models. Both models showed an internally validated adequate prognostic accuracy from 4 years follow-up (AUC 0.70 for both models) and good calibration. A refined model was adapted in view of the new ICD-11 criteria on 307 subjects with polymorphic ATPD, showing fair prognostic accuracy at 4 years (AUC: stepwise 0.68; LASSO 0.70). This study presents the first clinically based prediction model internally validated to adequately predict long-term psychotic recurrence in individuals with ATPD. The model can be automatable in EHRs, supporting further external validations and refinements to improve its prognostic accuracy.

[1]  P. Fusar-Poli,et al.  Third external replication of an individualised transdiagnostic prediction model for the automatic detection of individuals at risk of psychosis using electronic health records , 2021, Schizophrenia Research.

[2]  R. Stewart,et al.  Using Natural Language Processing on Electronic Health Records to Enhance Detection and Prediction of Psychosis Risk. , 2020, Schizophrenia bulletin.

[3]  C. Correll,et al.  Annual Research Review: Prevention of psychosis in adolescents - systematic review and meta-analysis of advances in detection, prognosis and intervention. , 2020, Journal of child psychology and psychiatry, and allied disciplines.

[4]  A. Danese,et al.  Implementing Precision Psychiatry: A Systematic Review of Individualized Prediction Models for Clinical Practice , 2020, Schizophrenia bulletin.

[5]  P. McGuire,et al.  Transdiagnostic individualized clinically-based risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: external replication in 2,430,333 US patients , 2020, Translational Psychiatry.

[6]  R. Dobson,et al.  Real-world implementation of precision psychiatry: Transdiagnostic risk calculator for the automatic detection of individuals at-risk of psychosis , 2020, Schizophrenia Research.

[7]  F. Schultze-Lutter,et al.  Effects of age and sex on clinical high-risk for psychosis in the community , 2020, World journal of psychiatry.

[8]  Richard Dobson,et al.  Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack. , 2020, Journal of visualized experiments : JoVE.

[9]  A. Meyer-Lindenberg,et al.  Prevention of Psychosis: Advances in Detection, Prognosis, and Intervention. , 2020, JAMA psychiatry.

[10]  Maarten van Smeden,et al.  Calibration: the Achilles heel of predictive analytics , 2019, BMC Medicine.

[11]  B. Crespo-Facorro,et al.  The prognostic role of catatonia, hallucinations, and symptoms of schizophrenia in acute and transient psychosis , 2019, Acta psychiatrica Scandinavica.

[12]  Andrew J. Vickers,et al.  A simple, step-by-step guide to interpreting decision curve analysis , 2019, Diagnostic and Prognostic Research.

[13]  P. McGuire,et al.  Unmet needs for treatment in 102 individuals with brief and limited intermittent psychotic symptoms (BLIPS): implications for current clinical recommendations , 2019, Epidemiology and Psychiatric Sciences.

[14]  A. Riecher-Rössler,et al.  Development and Validation of a Dynamic Risk Prediction Model to Forecast Psychosis Onset in Patients at Clinical High Risk. , 2019, Schizophrenia bulletin.

[15]  M. Ruiz-Veguilla,et al.  Predictors of diagnostic stability in acute and transient psychotic disorders: validation of previous findings and implications for ICD-11 , 2019, European Archives of Psychiatry and Clinical Neuroscience.

[16]  R. Dobson,et al.  Real World Implementation of a Transdiagnostic Risk Calculator for the Automatic Detection of Individuals at Risk of Psychosis in Clinical Routine: Study Protocol , 2019, Front. Psychiatry.

[17]  R. Murray,et al.  Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence , 2019, Schizophrenia bulletin.

[18]  P. McGuire,et al.  Unmet needs in patients with brief psychotic disorders: Too ill for clinical high risk services and not ill enough for first episode services , 2019, European Psychiatry.

[19]  Peter Tyrer,et al.  Innovations and changes in the ICD‐11 classification of mental, behavioural and neurodevelopmental disorders , 2019, World psychiatry : official journal of the World Psychiatric Association.

[20]  Ewout W. Steyerberg,et al.  The Science of Prognosis in Psychiatry: A Review , 2018, JAMA psychiatry.

[21]  S. Rauch,et al.  Digital devices and continuous telemetry: opportunities for aligning psychiatry and neuroscience , 2018, Neuropsychopharmacology.

[22]  P. McGuire,et al.  Long term outcomes of acute and transient psychotic disorders: The missed opportunity of preventive interventions , 2018, European Psychiatry.

[23]  S. Smesny,et al.  Clinical trajectories in the ultra-high risk for psychosis population , 2018, Schizophrenia Research.

[24]  P. McGuire,et al.  Transdiagnostic Risk Calculator for the Automatic Detection of Individuals at Risk and the Prediction of Psychosis: Second Replication in an Independent National Health Service Trust , 2018, Schizophrenia bulletin.

[25]  R. Murray,et al.  What causes psychosis? An umbrella review of risk and protective factors , 2018, World psychiatry : official journal of the World Psychiatric Association.

[26]  J. Kane,et al.  Improving outcomes of first‐episode psychosis: an overview , 2017, World psychiatry : official journal of the World Psychiatric Association.

[27]  P. Fusar-Poli,et al.  Diagnostic validity of ICD-10 acute and transient psychotic disorders and DSM-5 brief psychotic disorder , 2017, European Psychiatry.

[28]  P. McGuire,et al.  Long-term validity of the At Risk Mental State (ARMS) for predicting psychotic and non-psychotic mental disorders , 2017, European Psychiatry.

[29]  Paolo Fusar-Poli,et al.  Development and Validation of a Clinically Based Risk Calculator for the Transdiagnostic Prediction of Psychosis , 2017, JAMA psychiatry.

[30]  R. Dobson,et al.  Natural language processing to extract symptoms of severe mental illness from clinical text: the Clinical Record Interactive Search Comprehensive Data Extraction (CRIS-CODE) project , 2017, BMJ Open.

[31]  P. McGuire,et al.  Diagnostic and Prognostic Significance of Brief Limited Intermittent Psychotic Symptoms (BLIPS) in Individuals at Ultra High Risk , 2016, Schizophrenia bulletin.

[32]  G. Galeazzi,et al.  Acute and transient psychoses: clinical and nosological issues , 2016, BJPsych Advances.

[33]  W. Verhoeven,et al.  Cycloid psychoses in the psychosis spectrum: evidence for biochemical differences with schizophrenia , 2016, Neuropsychiatric disease and treatment.

[34]  Matthew Hotopf,et al.  Can mental health diagnoses in administrative data be used for research? A systematic review of the accuracy of routinely collected diagnoses , 2016, BMC Psychiatry.

[35]  P. McGuire,et al.  Diagnostic Stability of ICD/DSM First Episode Psychosis Diagnoses: Meta-analysis , 2016, Schizophrenia bulletin.

[36]  Andrea C. Fernandes,et al.  Cohort profile of the South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLaM BRC) Case Register: current status and recent enhancement of an Electronic Mental Health Record-derived data resource , 2016, BMJ Open.

[37]  Marcia K. Johnson,et al.  Cross-trial prediction of treatment outcome in depression: a machine learning approach. , 2016, The lancet. Psychiatry.

[38]  S. Lawrie,et al.  Prognosis of Brief Psychotic Episodes: A Meta-analysis. , 2016, JAMA psychiatry.

[39]  R. Tabarés-Seisdedos,et al.  Diagnosis and neurocognitive profiles in first-episode non-affective psychosis patients , 2016, European Archives of Psychiatry and Clinical Neuroscience.

[40]  L. Smeeth,et al.  The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement , 2015, PLoS medicine.

[41]  Tyrone D. Cannon,et al.  Specificity of Incident Diagnostic Outcomes in Patients at Clinical High Risk for Psychosis. , 2015, Schizophrenia bulletin.

[42]  G. Collins,et al.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement , 2015, BMC medicine.

[43]  O. Fawole,et al.  Acute and transient psychotic disorder in a developing country , 2014, The International journal of social psychiatry.

[44]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[45]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[46]  Graham Thornicroft,et al.  The South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLAM BRC) case register: development and descriptive data , 2009, BMC psychiatry.

[47]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: Developing a prognostic model , 2009, BMJ : British Medical Journal.

[48]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Epidemiology.

[49]  Guoqing Diao,et al.  Estimation of time‐dependent area under the ROC curve for long‐term risk prediction , 2006, Statistics in medicine.

[50]  A. Marneros,et al.  Longitudinal follow-up in acute and transient psychotic disorders and schizophrenia. , 2005, The British journal of psychiatry : the journal of mental science.

[51]  Rodney X. Sturdivant,et al.  Applied Logistic Regression: Hosmer/Applied Logistic Regression , 2005 .

[52]  P. Jørgensen,et al.  Acute and transient psychotic disorder: a 1‐year follow‐up study , 1997, Acta psychiatrica Scandinavica.

[53]  E. Susser,et al.  Epidemiology of nonaffective acute remitting psychosis vs schizophrenia. Sex and sociocultural setting. , 1994, Archives of general psychiatry.

[54]  R. R. Hocking The analysis and selection of variables in linear regression , 1976 .

[55]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[56]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .