Accuracy of diagnostic classification algorithms using cognitive-, electrophysiological-, and neuroanatomical data in antipsychotic-naïve schizophrenia patients

Abstract Background A wealth of clinical studies have identified objective biomarkers, which separate schizophrenia patients from healthy controls on a group level, but current diagnostic systems solely include clinical symptoms. In this study, we investigate if machine learning algorithms on multimodal data can serve as a framework for clinical translation. Methods Forty-six antipsychotic-naïve, first-episode schizophrenia patients and 58 controls underwent neurocognitive tests, electrophysiology, and magnetic resonance imaging (MRI). Patients underwent clinical assessments before and after 6 weeks of antipsychotic monotherapy with amisulpride. Nine configurations of different supervised machine learning algorithms were applied to first estimate the unimodal diagnostic accuracy, and next to estimate the multimodal diagnostic accuracy. Finally, we explored the predictability of symptom remission. Results Cognitive data significantly classified patients from controls (accuracies = 60–69%; p values = 0.0001–0.009). Accuracies of electrophysiology, structural MRI, and diffusion tensor imaging did not exceed chance level. Multimodal analyses with cognition plus any combination of one or more of the remaining three modalities did not outperform cognition alone. None of the modalities predicted symptom remission. Conclusions In this multivariate and multimodal study in antipsychotic-naïve patients, only cognition significantly discriminated patients from controls, and no modality appeared to predict short-term symptom remission. Overall, these findings add to the increasing call for cognition to be included in the definition of schizophrenia. To bring about the full potential of machine learning algorithms in first-episode, antipsychotic-naïve schizophrenia patients, careful a priori variable selection based on independent data as well as inclusion of other modalities may be required.

[1]  Paola Dazzan,et al.  Neuroimaging biomarkers to predict treatment response in schizophrenia: the end of 30 years of solitude? , 2014, Dialogues in clinical neuroscience.

[2]  M. Mallar Chakravarty,et al.  Can we accurately classify schizophrenia patients from healthy controls using magnetic resonance imaging and machine learning? A multi-method and multi-dataset study , 2017, Schizophrenia Research.

[3]  Randy L. Gollub,et al.  Neuropsychological Testing and Structural Magnetic Resonance Imaging as Diagnostic Biomarkers Early in the Course of Schizophrenia and Related Psychoses , 2011, Neuroinformatics.

[4]  P. Falkai,et al.  Detecting Neuroimaging Biomarkers for Schizophrenia: A Meta-Analysis of Multivariate Pattern Recognition Studies , 2015, Neuropsychopharmacology.

[5]  Benedicto Crespo-Facorro,et al.  Caudate nucleus volume and its clinical and cognitive correlations in first episode schizophrenia , 2007, Schizophrenia Research.

[6]  S. Kapur,et al.  Improvement of brain reward abnormalities by antipsychotic monotherapy in schizophrenia. , 2012, Archives of general psychiatry.

[7]  D L Braff,et al.  Sensorimotor gating and schizophrenia. Human and animal model studies. , 1990, Archives of general psychiatry.

[8]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[9]  Florin C. Popescu,et al.  Neurocognitive pattern analysis reveals classificatory hierarchy of attention deficits in schizophrenia. , 2014, Schizophrenia bulletin.

[10]  M. Kubicki,et al.  Review of functional and anatomical brain connectivity findings in schizophrenia , 2013, Current opinion in psychiatry.

[11]  Vaughan J. Carr,et al.  Systematic meta-review and quality assessment of the structural brain alterations in schizophrenia , 2012, Neuroscience & Biobehavioral Reviews.

[12]  Andres Hoyos Idrobo,et al.  Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines , 2016, NeuroImage.

[13]  E. Rostrup,et al.  Frontal fasciculi and psychotic symptoms in antipsychotic-naive patients with schizophrenia before and after 6 weeks of selective dopamine D2/3 receptor blockade. , 2016, Journal of psychiatry & neuroscience : JPN.

[14]  S. Andrews,et al.  Mismatch negativity: An index of a preattentive processing deficit in schizophrenia , 1991, Biological Psychiatry.

[15]  C. Svarer,et al.  Striatal D2/3 Binding Potential Values in Drug-Naïve First-Episode Schizophrenia Patients Correlate With Treatment Outcome , 2015, Schizophrenia bulletin.

[16]  E. Rostrup,et al.  Alterations of Intrinsic Connectivity Networks in Antipsychotic-Naïve First-Episode Schizophrenia , 2018, Schizophrenia bulletin.

[17]  Assen Jablensky,et al.  Psychiatric classifications: validity and utility , 2016, World psychiatry : official journal of the World Psychiatric Association.

[18]  S. Targum,et al.  The clinical global impressions scale: applying a research tool in clinical practice. , 2007, Psychiatry (Edgmont (Pa. : Township)).

[19]  H. Nelson,et al.  Dementia: The Estimation of Premorbid Intelligence Levels Using the New Adult Reading Test , 1978, Cortex.

[20]  L. K. Hansen,et al.  Two subgroups of antipsychotic-naive, first-episode schizophrenia patients identified with a Gaussian mixture model on cognition and electrophysiology , 2017, Translational Psychiatry.

[21]  D. Rujescu,et al.  Associations between psychosis endophenotypes across brain functional, structural, and cognitive domains , 2017, Psychological Medicine.

[22]  S. Kapur,et al.  Alterations of the Brain Reward System in Antipsychotic Naïve Schizophrenia Patients , 2012, Biological Psychiatry.

[23]  C. Haenschel,et al.  Core dysfunction in schizophrenia: electrophysiology trait biomarkers , 2012, Acta psychiatrica Scandinavica.

[24]  Nikolaj Bak,et al.  Data Driven Estimation of Imputation Error—A Strategy for Imputation with a Reject Option , 2016, PloS one.

[25]  T. Brugha,et al.  SCAN. Schedules for Clinical Assessment in Neuropsychiatry. , 1990, Archives of general psychiatry.

[26]  A. Storkey,et al.  Improved individualized prediction of schizophrenia in subjects at familial high risk, based on neuroanatomical data, schizotypal and neurocognitive features , 2017, Schizophrenia Research.

[27]  Florin C. Popescu,et al.  Impact of sample size and matching on single-subject classification of schizophrenia: A meta-analysis , 2017, Schizophrenia Research.

[28]  Qiyong Gong,et al.  Support vector machine-based classification of first episode drug-naïve schizophrenia patients and healthy controls using structural MRI , 2019, Schizophrenia Research.

[29]  T. Insel,et al.  Wesleyan University From the SelectedWorks of Charles A . Sanislow , Ph . D . 2010 Research Domain Criteria ( RDoC ) : Toward a New Classification Framework for Research on Mental Disorders , 2018 .

[30]  E. Bora,et al.  Social cognition in schizophrenia in comparison to bipolar disorder: A meta-analysis , 2016, Schizophrenia Research.

[31]  Daniel Rueckert,et al.  Tract-based spatial statistics: Voxelwise analysis of multi-subject diffusion data , 2006, NeuroImage.

[32]  Massimo Filippi,et al.  A selective review of structural connectivity abnormalities of schizophrenic patients at different stages of the disease , 2015, Schizophrenia Research.

[33]  Martin P Paulus,et al.  Trait contributions of complex dysregulated behavioral organization in schizophrenic patients , 2001, Biological Psychiatry.

[34]  R. Kahn,et al.  Schizophrenia is a cognitive illness: time for a change in focus. , 2013, JAMA psychiatry.

[35]  Mark W. Woolrich,et al.  FSL , 2012, NeuroImage.

[36]  Davide Castelvecchi,et al.  Can we open the black box of AI? , 2016, Nature.

[37]  Raquel E Gur,et al.  The Consortium on the Genetics of Schizophrenia: neurocognitive endophenotypes. , 2006, Schizophrenia bulletin.

[38]  Umberto Castellani,et al.  Machine Learning Approaches: From Theory to Application in Schizophrenia , 2013, Comput. Math. Methods Medicine.

[39]  K. Cheng,et al.  A Correlative Classification Study of Schizophrenic Patients with Results of Clinical Evaluation and Structural Magnetic Resonance Images , 2016, Behavioural neurology.

[40]  R Freedman,et al.  Neurophysiological evidence for a defect in neuronal mechanisms involved in sensory gating in schizophrenia. , 1982, Biological psychiatry.

[41]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[42]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[43]  S. Shaw,et al.  Trends in the epidemiology of alcoholism , 1978, Psychological Medicine.

[44]  P. Rabbitt,et al.  Cambridge Neuropsychological Test Automated Battery (CANTAB): a factor analytic study of a large sample of normal elderly volunteers. , 1994, Dementia.

[45]  B. Glenthøj,et al.  Effects of Blocking D2/D3 Receptors on Mismatch Negativity and P3a Amplitude of Initially Antipsychotic Naïve, First Episode Schizophrenia Patients , 2015, The international journal of neuropsychopharmacology.

[46]  Steven C. R. Williams,et al.  Using genetic, cognitive and multi-modal neuroimaging data to identify ultra-high-risk and first-episode psychosis at the individual level , 2013, Psychological Medicine.

[47]  D. Weinberger,et al.  Remission in schizophrenia: proposed criteria and rationale for consensus. , 2005, The American journal of psychiatry.

[48]  Philip D. Harvey,et al.  The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery , 2004, Schizophrenia Research.

[49]  R. Kahn,et al.  Brain volumes in schizophrenia: a meta-analysis in over 18 000 subjects. , 2013, Schizophrenia bulletin.

[50]  E. Rostrup,et al.  Cortical structures and their clinical correlates in antipsychotic-naïve schizophrenia patients before and after 6 weeks of dopamine D2/3 receptor antagonist treatment , 2018, Psychological Medicine.

[51]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[52]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[53]  S. Kay,et al.  The positive and negative syndrome scale (PANSS) for schizophrenia. , 1987, Schizophrenia bulletin.

[54]  Ninon Burgos,et al.  New advances in the Clinica software platform for clinical neuroimaging studies , 2019 .

[55]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[56]  P. Falkai,et al.  Consensus paper of the WFSBP Task Force on Biological Markers: Criteria for biomarkers and endophenotypes of schizophrenia part I: Neurophysiology , 2015, The world journal of biological psychiatry : the official journal of the World Federation of Societies of Biological Psychiatry.

[57]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[58]  T. Insel,et al.  Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? , 2012, Molecular Psychiatry.

[59]  Cassandra J. Lowe,et al.  The neurocognitive consequences of sleep restriction: A meta-analytic review , 2017, Neuroscience & Biobehavioral Reviews.

[60]  S. Lui,et al.  A Selective Review of Cerebral Abnormalities in Patients With First-Episode Schizophrenia Before and After Treatment. , 2016, The American journal of psychiatry.

[61]  S. Faraone,et al.  Neurocognition in first-episode schizophrenia: a meta-analytic review. , 2009, Neuropsychology.

[62]  B. Glenthøj,et al.  Effects of Dopamine D2/D3 Blockade on Human Sensory and Sensorimotor Gating in Initially Antipsychotic-Naive, First-Episode Schizophrenia Patients , 2014, Neuropsychopharmacology.

[63]  William Stafford Noble,et al.  Support vector machine , 2013 .

[64]  Luis M. San-José-Revuelta,et al.  A Computer-Aided Diagnosis System With EEG Based on the P3b Wave During an Auditory Odd-Ball Task in Schizophrenia , 2017, IEEE Transactions on Biomedical Engineering.