Multi-modal deep learning model for auxiliary diagnosis of Alzheimer's disease

Abstract Alzheimer’s disease (AD) is one of the most difficult to cure diseases. Alzheimer’s disease seriously affects the normal lives of the elderly and their families. The mild cognitive impairment (MCI) is a transitional state between the normal aging and Alzheimer’s disease, and MCI is most likely to converted to AD later. MCI is often misdiagnosed as the symptoms of normal aging, which results to miss the best opportunity of treatment. Therefore, the accurate diagnosis of MCI is essential for the early diagnosis and treatment of AD. This paper presents a deep learning model for the auxiliary diagnosis of AD, which simulates the clinician’s diagnostic process. During the diagnosis of AD, clinician usually refers to the results of various neuroimaging, as well as the results of neuropsychological diagnosis. In this paper, the multi-modal medical images are trained by two independent convolutional neural networks. Then the consistency of the output of two convolutional neural networks is judged by the correlation analysis. Finally, the results of multi-modal neuroimaging diagnosis are combined with the results of clinical neuropsychological diagnosis. The proposed model provides a comprehensive analysis about patient’s pathology and psychology at the same time, therefore, it improves the accuracy of auxiliary diagnosis. The diagnosis process is closer to the process of clinician’s diagnosis and easy to implement. Experiments on the public database of ADNI (Alzheimer’s disease neuroimaging initiative) show that the proposed method has better performance, and can achieve an excellent diagnostic efficiency in the auxiliary diagnosis of AD.

[1]  Tara N. Sainath,et al.  Deep Convolutional Neural Networks for Large-scale Speech Tasks , 2015, Neural Networks.

[2]  Daoqiang Zhang,et al.  Multimodal classification of Alzheimer's disease and mild cognitive impairment , 2011, NeuroImage.

[3]  Manuel Graña,et al.  A lattice computing approach to Alzheimer's disease computer assisted diagnosis based on MRI data , 2015, Neurocomputing.

[4]  R. M. Keegan,et al.  Recent advances in the neurobiology and neuropharmacology of Alzheimer's disease. , 2018, Biomedicine & pharmacotherapy = Biomedecine & pharmacotherapie.

[5]  Dinggang Shen,et al.  View‐aligned hypergraph learning for Alzheimer's disease diagnosis with incomplete multi‐modality data , 2017, Medical Image Anal..

[6]  Ricardo Nitrini,et al.  Probing the correlation of neuronal loss, neurofibrillary tangles, and cell death markers across the Alzheimer's disease Braak stages: a quantitative study in humans , 2018, Neurobiology of Aging.

[7]  Dae-Ki Kang,et al.  Biased Dropout and Crossmap Dropout: Learning towards effective Dropout regularization in convolutional neural network , 2018, Neural Networks.

[8]  Tao Mei,et al.  Deep Collaborative Embedding for Social Image Understanding , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[12]  Prospero C. Naval,et al.  DemNet: A Convolutional Neural Network for the detection of Alzheimer's Disease and Mild Cognitive Impairment , 2016, 2016 IEEE Region 10 Conference (TENCON).

[13]  Ghassem Tofighi,et al.  Classification of Alzheimer's Disease Structural MRI Data by Deep Learning Convolutional Neural Networks , 2016, ArXiv.

[14]  Qian Wang,et al.  Deep embedding convolutional neural network for synthesizing CT image from T1‐Weighted MR image , 2018, Medical Image Anal..

[15]  Markus Otto,et al.  Capillary cerebral amyloid angiopathy in Alzheimer’s disease: association with allocortical/hippocampal microinfarcts and cognitive decline , 2018, Acta Neuropathologica.

[16]  Johannes Schröder,et al.  Neuropsychological profiles and verbal abilities in lifelong bilinguals with mild cognitive impairment and Alzheimer's disease. , 2015, Journal of Alzheimer's disease : JAD.

[17]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[18]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[19]  Yorghos Tripodis,et al.  Mini Mental State Examination and Logical Memory scores for entry into Alzheimer’s disease trials , 2016, Alzheimer's Research & Therapy.

[20]  Liang Chen,et al.  Multi-modal classification of Alzheimer's disease using nonlinear graph fusion , 2017, Pattern Recognit..

[21]  Maode Ma,et al.  Intelligent Image Recognition System for Marine Fouling Using Softmax Transfer Learning and Deep Convolutional Neural Networks , 2017, Complex..

[22]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[23]  Youbao Tang,et al.  Scene Text Detection and Segmentation Based on Cascaded Convolution Neural Networks , 2017, IEEE Transactions on Image Processing.

[24]  Dinggang Shen,et al.  A Robust Deep Model for Improved Classification of AD/MCI Patients , 2015, IEEE Journal of Biomedical and Health Informatics.

[25]  Sidong Liu,et al.  Multimodal Neuroimaging Feature Learning for Multiclass Diagnosis of Alzheimer's Disease , 2015, IEEE Transactions on Biomedical Engineering.

[26]  Daoqiang Zhang,et al.  Hierarchical fusion of features and classifier decisions for Alzheimer's disease diagnosis , 2014, Human brain mapping.

[27]  ZhiFei Dai,et al.  Applications, opportunities and challenges of molecular probes in the diagnosis and treatment of major diseases , 2017 .

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Christine Preibisch,et al.  Alzheimer Disease and Mild Cognitive Impairment: Integrated Pulsed Arterial Spin-Labeling MRI and 18F-FDG PET. , 2018, Radiology.

[30]  Dinggang Shen,et al.  A Hybrid of Deep Network and Hidden Markov Model for MCI Identification with Resting-State fMRI , 2015, MICCAI.

[31]  Jinhui Tang,et al.  Weakly Supervised Deep Matrix Factorization for Social Image Understanding , 2017, IEEE Transactions on Image Processing.

[32]  Shihui Ying,et al.  Multimodal Neuroimaging Feature Learning With Multimodal Stacked Deep Polynomial Networks for Diagnosis of Alzheimer's Disease , 2018, IEEE Journal of Biomedical and Health Informatics.

[33]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[34]  A. Fagan,et al.  Multimodal techniques for diagnosis and prognosis of Alzheimer's disease , 2009, Nature.

[35]  Wei Gao,et al.  Automated detection of pathologic white matter alterations in Alzheimer's disease using combined diffusivity and kurtosis method , 2017, Psychiatry Research: Neuroimaging.

[36]  Ghassem Tofighi,et al.  Deep Learning-based Pipeline to Recognize Alzheimer’s Disease using fMRI Data , 2016, bioRxiv.

[37]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[38]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[39]  Marios Anthimopoulos,et al.  Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[40]  Seong-Whan Lee,et al.  Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis , 2014, NeuroImage.

[41]  Carlo Caltagirone,et al.  Neuropsychological correlates of cognitive, emotional-affective and auto-activation apathy in Alzheimer's disease , 2018, Neuropsychologia.

[42]  G. Wong,et al.  Human amyloid β peptide and tau co-expression impairs behavior and causes specific gene expression changes in Caenorhabditis elegans , 2018, Neurobiology of Disease.

[43]  Quanzheng Li,et al.  Early Diagnosis of Alzheimer's Disease Based on Resting-State Brain Networks and Deep Learning , 2019, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[44]  Giovanni Montana,et al.  Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks , 2015, ICPRAM 2015.

[45]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[46]  Taesung Park,et al.  Estimation of Area Under the ROC Curve under nonignorable verification bias. , 2018, Statistica Sinica.