Predicting Risk of Suicide Attempts Over Time Through Machine Learning

Traditional approaches to the prediction of suicide attempts have limited the accuracy and scale of risk detection for these dangerous behaviors. We sought to overcome these limitations by applying machine learning to electronic health records within a large medical database. Participants were 5,167 adult patients with a claim code for self-injury (i.e., ICD-9, E95x); expert review of records determined that 3,250 patients made a suicide attempt (i.e., cases), and 1,917 patients engaged in self-injury that was nonsuicidal, accidental, or nonverifiable (i.e., controls). We developed machine learning algorithms that accurately predicted future suicide attempts (AUC = 0.84, precision = 0.79, recall = 0.95, Brier score = 0.14). Moreover, accuracy improved from 720 days to 7 days before the suicide attempt, and predictor importance shifted across time. These findings represent a step toward accurate and scalable risk detection and provide insight into how suicide attempt risk shifts over time.

[1]  On Safari , 1965 .

[2]  [On suicide and suicidal attempts]. , 1976, ha-Ahot be-Yisrael.

[3]  J. J. Narraway,et al.  Probability machines , 1989, Microprocess. Microprogramming.

[4]  J. Neeleman A continuum of premature death. Meta-analysis of competing mortality in the psychosocially vulnerable. , 2001, International journal of epidemiology.

[5]  Gopal K. Singh,et al.  Area deprivation and widening inequalities in US mortality, 1969-1998. , 2003, American journal of public health.

[6]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[7]  Stephen E. Robertson,et al.  Understanding inverse document frequency: on theoretical arguments for IDF , 2004, J. Documentation.

[8]  Lisa I. Iezzoni,et al.  Risk Adjustment of Medicare Capitation Payments Using the CMS-HCC Model , 2004, Health care financing review.

[9]  Matthew K Nock,et al.  The American Association of Suicidology Warning Signs for Suicide : Theory , Research , and Clinical Applications , 2006 .

[10]  S. Kotsiantis Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[11]  Brian E. Granger,et al.  IPython: A System for Interactive Scientific Computing , 2007, Computing in Science & Engineering.

[12]  A. Wenzel,et al.  A cognitive model of suicidal behavior: Theory and treatment , 2008 .

[13]  C. Waternaux,et al.  Classification trees distinguish suicide attempters in major psychiatric disorders: a model of clinical decision making. , 2008, The Journal of clinical psychiatry.

[14]  D. Roden,et al.  Development of a Large‐Scale De‐Identified DNA Biobank to Enable Personalized Medicine , 2008, Clinical pharmacology and therapeutics.

[15]  Andreas Ziegler,et al.  On safari to Random Jungle: a fast implementation of Random Forests for high-dimensional data , 2010, Bioinform..

[16]  Matthew K Nock,et al.  Attentional bias toward suicide-related stimuli predicts suicidal behavior. , 2010, Journal of abnormal psychology.

[17]  Son Doan,et al.  Application of information technology: MedEx: a medication information extraction system for clinical narratives , 2010, J. Am. Medical Informatics Assoc..

[18]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[19]  Andreas Ziegler,et al.  On safari to Random Jungle: a fast implementation of Random Forests for high-dimensional data , 2010, Bioinform..

[20]  E. Steyerberg,et al.  [Regression modeling strategies]. , 2011, Revista espanola de cardiologia.

[21]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[22]  R. O’Connor Towards an integrated motivational-volitional model of suicidal behaviour. , 2011 .

[23]  Jörg Drechsler,et al.  Multiple Imputation for Nonresponse , 2011 .

[24]  T. Joiner,et al.  Sleep problems outperform depression and hopelessness as cross-sectional and longitudinal predictors of suicidal ideation and behavior in young adults in the military. , 2012, Journal of affective disorders.

[25]  Pramil Cheriyath,et al.  A prediction model for COPD readmissions: catching up, catching our breath, and improving a national problem , 2012, Journal of community hospital internal medicine perspectives.

[26]  David Delgado-Gómez,et al.  Suicide attempters classification: Toward predictive models of suicidal behavior , 2012, Neurocomputing.

[27]  Ewout W Steyerberg,et al.  Regression trees for predicting mortality in patients with cardiovascular disease: What improvement is achieved by using ensemble-based methods? , 2012, Biometrical journal. Biometrische Zeitschrift.

[28]  W. J. Boscardin,et al.  Estimating Harrell's Optimism on Predictive Indices Using Bootstrap Samples , 2013 .

[29]  M. Pencina,et al.  Evaluation of Markers and Risk Prediction Models , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  Patrick Royston,et al.  Correcting for Optimistic Prediction in Small Data Sets , 2014, American journal of epidemiology.

[32]  Evan M. Kleiman,et al.  Meta-analysis of risk factors for nonsuicidal self-injury. , 2015, Clinical psychology review.

[33]  Evan M. Kleiman,et al.  Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: a meta-analysis of longitudinal studies , 2015, Psychological Medicine.

[34]  Joseph Futoma,et al.  A comparison of models for predicting early hospital readmissions , 2015, J. Biomed. Informatics.

[35]  Evan M. Kleiman,et al.  Letter to the Editor: Suicide as a complex classification problem: machine learning and related techniques can advance suicide prediction - a reply to Roaldset (2016) , 2016, Psychological Medicine.

[36]  Tianxi Cai,et al.  Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports , 2015, Molecular Psychiatry.

[37]  Evan M. Kleiman,et al.  Anxiety and its disorders as risk factors for suicidal thoughts and behaviors: A meta-analytic review. , 2016, Clinical psychology review.

[38]  Evan M. Kleiman,et al.  Biological risk factors for suicidal behaviors: a meta-analysis , 2016, Translational psychiatry.

[39]  D. Shepard,et al.  Suicide and Suicidal Attempts in the United States: Costs and Policy Implications , 2015, Suicide & life-threatening behavior.

[40]  Andreas Ziegler,et al.  ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R , 2015, 1508.04409.

[41]  Ben Y. Reis,et al.  Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) , 2016, Molecular Psychiatry.

[42]  Evan M. Kleiman,et al.  Risk Factors for Suicidal Thoughts and Behaviors: A Meta-Analysis of 50 Years of Research , 2017, Psychological bulletin.

[43]  Douglas G. Altman,et al.  Analysing Data and Undertaking Meta‐Analyses , 2019 .