Bilinear pooling and metric learning network for early Alzheimer's disease identification with FDG-PET images

18F-fluorodeoxyglucose (FDG)-positron emission tomography (PET) reveals altered brain metabolism in individuals with mild cognitive impairment (MCI) and Alzheimer’s disease (AD). Some biomarkers derived from FDG-PET by computer-aided-diagnosis (CAD) technologies have been proved that they can accurately diagnosis normal control (NC), MCI, and AD. However, the studies of identification of early MCI (EMCI) and late MCI (LMCI) with FDG-PET images are still insufficient. Compared with studies based on fMRI and DTI images, the researches of the inter-region representation features in FDG-PET images are insufficient. Moreover, considering the variability in different individuals, some hard samples which are very similar with both two classes limit the classification performance. To tackle these problems, in this paper, we propose a novel bilinear pooling and metric learning network (BMNet), which can extract the inter-region representation features and distinguish hard samples by constructing embedding space. To validate the proposed method, we collect 998 FDG-PET images from Alzheimer's disease neuroimaging initiative (ADNI) including 263 normal control (NC) patients, 290 EMCI patients, 147 LMCI patients, and 198 AD patients. Following the common preprocessing steps, 90 features are extracted from each FDG-PET image according to the automatic anatomical landmark (AAL) template and then sent into the proposed network. Extensive 5-fold cross-validation experiments are performed for multiple two-class classifications. Experiments show that most metrics are improved after adding the bilinear pooling module and metric losses to the Baseline model respectively. Specifically, in the classification task between EMCI and LMCI, the specificity improves 6.38% after adding the triple metric loss, and the negative predictive value (NPV) improves 3.45% after using the bilinear pooling module. In addition, the accuracy of classification between EMCI and LMCI achieves 79.64% using imbalanced FDG-PET images, which illustrates that the proposed method yields a state-of-the-art result of the classification accuracy between EMCI and LMCI based on PET images.

[1]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[2]  Xiaohui Yao,et al.  Multi-modal neuroimaging feature selection with consistent metric constraint for diagnosis of Alzheimer's disease , 2019, Medical Image Anal..

[3]  Richard J. Caselli,et al.  Deep-learning-based classification of FDG-PET data for Alzheimer's disease categories , 2017, Symposium on Medical Information Processing and Analysis.

[4]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Jing Li,et al.  Learning brain connectivity of Alzheimer's disease by sparse inverse covariance estimation , 2010, NeuroImage.

[6]  Yunde Jia,et al.  Revisiting Bilinear Pooling: A Coding Perspective , 2020, AAAI.

[7]  Timo Grimmer,et al.  Metabolic connectivity for differential diagnosis of dementing disorders , 2017, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[8]  Daoqiang Zhang,et al.  Attention-Diffusion-Bilinear Neural Network for Brain Network Analysis , 2020, IEEE Transactions on Medical Imaging.

[9]  Gerta Rücker,et al.  Principal Components Analysis of Brain Metabolism Predicts Development of Alzheimer Dementia , 2018, The Journal of Nuclear Medicine.

[10]  R. Petersen,et al.  Mild cognitive impairment , 2006, The Lancet.

[11]  Zhaoyu Liu,et al.  The risk prediction of Alzheimer’s disease based on the deep learning model of brain 18F-FDG positron emission tomography , 2019, Saudi journal of biological sciences.

[12]  Vijaya L. Melnick,et al.  Alzheimer’s Dementia , 1985, Contemporary Issues in Biomedicine, Ethics, and Society.

[13]  Yinghuan Shi,et al.  MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT Prostate Segmentation via Online Sampling , 2020 .

[14]  2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) , 2018 .

[15]  Rasmus R. Paulsen,et al.  Deep metric learning for otitis media classification , 2021, Medical Image Anal..

[16]  Ee-Leng Tan,et al.  Self-weighted adaptive structure learning for ASD diagnosis via multi-template multi-center representation , 2020, Medical Image Anal..

[17]  Malek Adjouadi,et al.  Gaussian discriminative component analysis for early detection of Alzheimer’s disease: A supervised dimensionality reduction algorithm , 2020, Journal of Neuroscience Methods.

[18]  D. Shen,et al.  High-Order Laplacian Regularized Low-Rank Representation for Multimodal Dementia Diagnosis , 2021, Frontiers in Neuroscience.

[19]  Samuel Kadoury,et al.  Classification of Alzheimer's and MCI Patients from Semantically Parcelled PET Images: A Comparison between AV45 and FDG-PET , 2018, Int. J. Biomed. Imaging.

[20]  Malek Adjouadi,et al.  A Gaussian-based model for early detection of mild cognitive impairment using multimodal neuroimaging , 2019, Journal of Neuroscience Methods.

[21]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Alzheimer’s Association.  2018 Alzheimer's disease facts and figures , 2018, Alzheimer's & Dementia.

[23]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Yong He,et al.  Disrupted Functional Brain Connectome in Individuals at Risk for Alzheimer's Disease , 2013, Biological Psychiatry.

[25]  Evan M. Gordon,et al.  Local-Global Parcellation of the Human Cerebral Cortex From Intrinsic Functional Connectivity MRI , 2017, bioRxiv.

[26]  Alejandro F. Frangi,et al.  Graph convolution network with similarity awareness and adaptive calibration for disease-induced deterioration prediction , 2020, Medical Image Anal..

[27]  Kyong Hwan Jin,et al.  Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging , 2017, Behavioural Brain Research.

[28]  Alessandro Giuliani,et al.  Early identification of MCI converting to AD: a FDG PET study , 2017, European Journal of Nuclear Medicine and Molecular Imaging.

[29]  Ee-Leng Tan,et al.  Self-calibrated brain network estimation and joint non-convex multi-task learning for identification of early Alzheimer's disease , 2020, Medical Image Anal..

[30]  Hucheng Zhou,et al.  Dual-Model Radiomic Biomarkers Predict Development of Mild Cognitive Impairment Progression to Alzheimer’s Disease , 2019, Front. Neurosci..

[31]  C. Jack,et al.  Preclinical Alzheimer's disease: Definition, natural history, and diagnostic criteria , 2016, Alzheimer's & Dementia.

[32]  Malek Adjouadi,et al.  A Deep Neural Network Approach for Early Diagnosis of Mild Cognitive Impairment Using Multiple Features , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[33]  Danni Cheng,et al.  Classification of Alzheimer’s Disease by Combination of Convolutional and Recurrent Neural Networks Using FDG-PET Images , 2018, Front. Neuroinform..

[34]  Nanning Zheng,et al.  Person Re-identification by Multi-Channel Parts-Based CNN with Improved Triplet Loss Function , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Shibani Singh.  Deep Learning based Classification of FDG-PET Data for Alzheimer's Disease , 2017 .

[36]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[37]  Simone Lista,et al.  Dementia: The rising global tide of cognitive impairment , 2016, Nature Reviews Neurology.

[38]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Yang Liu,et al.  Use of a Sparse-Response Deep Belief Network and Extreme Learning Machine to Discriminate Alzheimer's Disease, Mild Cognitive Impairment, and Normal Controls Based on Amyloid PET/MRI Images , 2021, Frontiers in Medicine.

[40]  A. Rominger,et al.  Individual brain metabolic connectome indicator based on Kullback-Leibler Divergence Similarity Estimation predicts progression from mild cognitive impairment to Alzheimer’s dementia , 2020, European Journal of Nuclear Medicine and Molecular Imaging.

[41]  Gretel Sanabria-Diaz,et al.  Glucose Metabolism during Resting State Reveals Abnormal Brain Networks Organization in the Alzheimer’s Disease and Mild Cognitive Impairment , 2013, PloS one.

[42]  G. Rücker,et al.  Prognosis of conversion of mild cognitive impairment to Alzheimer's dementia by voxel-wise Cox regression based on FDG PET data , 2018, NeuroImage: Clinical.