Using Psychologically-Informed Priors for Suicide Prediction in the CLPsych 2021 Shared Task

This paper describes our approach to the CLPsych 2021 Shared Task, in which we aimed to predict suicide attempts based on Twitter feed data. We addressed this challenge by emphasizing reliance on prior domain knowledge. We engineered novel theory-driven features, and integrated prior knowledge with empirical evidence in a principled manner using Bayesian modeling. While this theory-guided approach increases bias and lowers accuracy on the training set, it was successful in preventing over-fitting. The models provided reasonable classification accuracy on unseen test data (0.68<=AUC<= 0.84). Our approach may be particularly useful in prediction tasks trained on a relatively small data set.

[1]  Lucila Ohno-Machado,et al.  Natural language processing: an introduction , 2011, J. Am. Medical Informatics Assoc..

[2]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[3]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[4]  M. Large,et al.  Known unknowns and unknown unknowns in suicide risk assessment: evidence from meta-analyses of aleatory and epistemic uncertainty , 2017, BJPsych bulletin.

[5]  K. Bretonnel Cohen,et al.  Sentiment Analysis of Suicide Notes: A Shared Task , 2012, Biomedical informatics insights.

[6]  J. Pirkis,et al.  Suicide and Suicide Prevention From a Global Perspective. , 2020, Crisis.

[7]  J. Gabry,et al.  Bayesian Applied Regression Modeling via Stan , 2016 .

[8]  Alex B. Fine,et al.  Natural Language Processing of Social Media as Screening for Suicide Risk , 2018, Biomedical informatics insights.

[9]  Almog Simchon,et al.  A Psychologically Informed Approach to CLPsych Shared Task 2018 , 2018, CLPsych@NAACL-HTL.

[10]  Enrique Baca-García,et al.  Novel Use of Natural Language Processing (NLP) to Predict Suicidal Ideation and Psychiatric Symptoms in a Text-Based Mental Health Intervention in Madrid , 2016, Comput. Math. Methods Medicine.

[11]  Ryan L. Boyd,et al.  The Development and Psychometric Properties of LIWC2015 , 2015 .

[12]  J. Pennebaker,et al.  Word Use in the Poetry of Suicidal and Nonsuicidal Poets , 2001, Psychosomatic medicine.

[13]  W. Caan,et al.  An overview of systematic reviews on the public health consequences of social isolation and loneliness. , 2017, Public health.

[14]  Jonathan Gemmell,et al.  Using Machine Learning Algorithms to Detect Suicide Risk Factors on Twitter , 2019, 2019 International Conference on Data Mining Workshops (ICDMW).

[15]  Glen Coppersmith,et al.  Exploratory Analysis of Social Media Prior to a Suicide Attempt , 2016, CLPsych@HLT-NAACL.

[16]  M. Naghavi Global, regional, and national burden of suicide mortality 1990 to 2016: systematic analysis for the Global Burden of Disease Study 2016 , 2019, BMJ.

[17]  Max Kuhn,et al.  caret: Classification and Regression Training , 2015 .

[18]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[19]  Barbara J. Grosz,et al.  Natural-Language Processing , 1982, Artificial Intelligence.

[20]  Richard T. Liu,et al.  Sleep and suicide: A systematic review and meta-analysis of longitudinal studies. , 2020, Clinical psychology review.

[21]  Maya Elin O'Neil,et al.  Suicide Risk Factors and Risk Assessment Tools: A Systematic Review , 2012 .

[22]  Holly Hedegaard,et al.  Increase in Suicide in the United States, 1999-2014. , 2016, NCHS data brief.

[23]  Stephanie K. Doupnik,et al.  Association of Suicide Prevention Interventions With Subsequent Suicide Attempts, Linkage to Follow-up Care, and Depression Symptoms for Acute Care Settings: A Systematic Review and Meta-analysis. , 2020, JAMA psychiatry.

[24]  M. Roca,et al.  Gender differences in suicidal behavior in adolescents and young adults: systematic review and meta-analysis of longitudinal studies , 2019, International Journal of Public Health.

[25]  Roi Reichart,et al.  Deep neural networks detect suicide risk from textual facebook posts , 2020, Scientific reports.

[26]  Evan M. Kleiman,et al.  Risk Factors for Suicidal Thoughts and Behaviors: A Meta-Analysis of 50 Years of Research , 2017, Psychological bulletin.

[27]  D. Asch,et al.  Facebook language predicts depression in medical records , 2018, Proceedings of the National Academy of Sciences.

[28]  J. Ribeiro,et al.  Depression and hopelessness as risk factors for suicide ideation, attempts and death: meta-analysis of longitudinal studies , 2018, British Journal of Psychiatry.

[29]  Andrew C. Porter,et al.  Annual Research Review: A meta-analytic review of worldwide suicide rates in adolescents. , 2019, Journal of child psychology and psychiatry, and allied disciplines.

[30]  Erik van Zwet,et al.  A Proposal for Informative Default Priors Scaled by the Standard Error of Estimates , 2020, The American Statistician.

[31]  M. Large,et al.  Can we usefully stratify patients according to suicide risk? , 2017, British Medical Journal.