Multi-modal deep learning for predicting progression of Alzheimer's disease using bi-linear shake fusion

Alzheimer’s Disease (AD) which causes declination of cognitive function is one of the most severe social issues in the world. It has already been known that AD cannot be cured and treatment can only delay its progression. Therefore, it is very important to detect AD in early stage and prevent it to be worse. Furthermore, sooner the progression is detected, better the prognosis will be. In this research, we developed a novel multi-modal deep learning method to predict conversion from Mild Cognitive Impairment (MCI), which is the stage between cognitively normal older people and AD. In our method, the multi-modal input data are defined as structural Magnetic Resonance Imaging (MRI) images and clinical data including several cognitive scores, APOE genotype, gender and age obtained from Alzheimer’s Disease Neuroimaging Initiative cohort (ADNI). Our criteria of selecting these input data are that they are mostly obtained by non-invasive examination. The proposed method integrates features obtained from MRI images and clinical data effectively by using bi-linear fusion. Bi-linear fusion computes the products of all elements between image and clinical features, where the correlation between them are included. That led to a big improvement of prediction accuracy in the experiment. The prediction model using bi-linear fusion achieved to predict conversion in one year with 0.86 accuracy, comparing with 0.76 accuracy using linear fusion. The proposed method is useful for screening examination for AD or deciding a stratification approach within clinical trials since it achieved a high accuracy while the input data is relatively easy to be obtained.

[1]  Sidong Liu,et al.  Multimodal Neuroimaging Feature Learning for Multiclass Diagnosis of Alzheimer's Disease , 2015, IEEE Transactions on Biomedical Engineering.

[2]  Nick C Fox,et al.  The Diagnosis of Mild Cognitive Impairment due to Alzheimer’s Disease: Recommendations from the National Institute on Aging-Alzheimer’s Association Workgroups on Diagnostic Guidelines for Alzheimer’s Disease , 2011 .

[3]  Matthieu Cord,et al.  BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection , 2019, AAAI.

[4]  Daoqiang Zhang,et al.  Domain Transfer Learning for MCI Conversion Prediction , 2012, MICCAI.

[5]  Matthieu Cord,et al.  MUTAN: Multimodal Tucker Fusion for Visual Question Answering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Isabel Santana,et al.  Prognosis of Early-Onset vs. Late-Onset Mild Cognitive Impairment: Comparison of Conversion Rates and Its Predictors , 2016, Geriatrics.

[7]  John Breitner,et al.  A highly predictive signature of cognition and brain atrophy for progression to Alzheimer's dementia , 2019, GigaScience.

[8]  G. Bloom.  Amyloid-β and tau: the trigger and bullet in Alzheimer disease pathogenesis. , 2014, JAMA neurology.

[9]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[10]  J. Haines,et al.  ApoE-4 and Age at Onset of Alzheimer's Disease , 1997, Neurology.

[11]  E. Tangalos,et al.  Mild Cognitive Impairment Clinical Characterization and Outcome , 1999 .

[12]  Sterling C. Johnson,et al.  Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images , 2017, ArXiv.

[13]  Daoqiang Zhang,et al.  Predicting Future Clinical Changes of MCI Patients Using Longitudinal and Multimodal Biomarkers , 2012, PloS one.

[14]  Sterling C. Johnson,et al.  Predicting Alzheimer’s disease progression using multi-modal deep learning approach , 2019, Scientific Reports.

[15]  T. Kurosaki,et al.  Measurement of functional activities in older adults in the community. , 1982, Journal of gerontology.

[16]  Pedro Rosa-Neto,et al.  Identifying incipient dementia individuals using machine learning and amyloid imaging , 2017, Neurobiology of Aging.

[17]  D. Rueckert,et al.  Structural brain imaging in Alzheimer’s disease and mild cognitive impairment: biomarker analysis and shared morphometry database , 2018, Scientific Reports.

[18]  Andre Altmann,et al.  Sex modifies the APOE‐related risk of developing Alzheimer disease , 2014, Annals of neurology.

[19]  Sidong Liu,et al.  Early diagnosis of Alzheimer's disease with deep learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).