Classification of Alzheimer Disease on Imaging Modalities with Deep CNNs Using Cross-Modal Transfer Learning

A recent imaging modality Diffusion Tensor Imaging completes information used from Structural MRI in studies of Alzheimer disease. A large number of recent studies has explored pathologic staging of Alzheimer disease using the Mean Diffusivity maps extracted from the Diffusion Tensor Imaging modality. The Deep Neural Networks are seducing tools for classification of subjects' imaging data in computer-aided diagnosis of Alzheimer's disease. The major problem here is the lack of a publicly available large amount of training data in both modalities. The lack number of training data yields over-fitting phenomena. We propose a method of a cross-modal transfer learning: from Structural MRI to Diffusion Tensor Imaging modality. Models pre-trained on a structural MRI dataset with domain-depended data augmentation are used as initialization of network parameters to train on Mean Diffusivity data. The method shows a reduction of the over-fitting phenomena, improves learning performance, and thus increases the accuracy of prediction. Classifiers are then fused by a majority vote resulting in augmented scores of classification between Normal Control, Alzheimer Patients and Mild Cognitive Impairment subjects on a subset of ADNI dataset.