Automated Verbal and Non-verbal Speech Analysis of Interviews of Individuals with Schizophrenia and Depression

Schizophrenia and depression are the two most common mental disorders associated with negative symptoms that contribute to poor functioning and quality of life for millions of patients globally. This study is part of a larger research project. The overall aim of the project is to develop an automated objective pipeline that aids clinical diagnosis and provides more insights into symptoms of mental illnesses. In our previous work, we have analyzed non-verbal cues and linguistic cues of individuals with schizophrenia. In this study, we extend our work to include participants with depression. Powered by natural language processing techniques, we extract verbal features, both dictionary-based and vector-based, from participants’ interviews that were automatically transcribed. We also extracted conversational, phonatory, articulatory and prosodic features from the interviews to understand the conversational and acoustic characteristics of schizophrenia and depression. Combining these features, we applied ensemble learning with leave-one-out cross-validation to classify healthy controls, schizophrenic and depressive patients, achieving an accuracy of 69%-75% in paired classification. From those same features, we also predict the subjective Negative Symptoms Assessment 16 scores of patients with schizophrenia or depression, yielding an accuracy of 90.5% for NSA2 but lower accuracy for other NSA indices. Our analysis also revealed significant linguistic and non-verbal differences that are potentially symptomatic of schizophrenia and depression respectively.