Landmark‐based deep multi‐instance learning for brain disease diagnosis

HighlightsWe developed a novel deep multi‐instance convolutional neural network to automatically learn both local and global representations for MR images.We proposed a landmark‐based image patch extraction approach based on a data‐driven landmark discovery algorithm.We trained the model on ADNI‐1 and tested it on two independent datasets (i.e., ADNI‐2 and MIRIAD). Graphical abstract Figure. No Caption available. Abstract In conventional Magnetic Resonance (MR) image based methods, two stages are often involved to capture brain structural information for disease diagnosis, i.e., 1) manually partitioning each MR image into a number of regions‐of‐interest (ROIs), and 2) extracting pre‐defined features from each ROI for diagnosis with a certain classifier. However, these pre‐defined features often limit the performance of the diagnosis, due to challenges in 1) defining the ROIs and 2) extracting effective disease‐related features. In this paper, we propose a landmark‐based deep multi‐instance learning (LDMIL) framework for brain disease diagnosis. Specifically, we first adopt a data‐driven learning approach to discover disease‐related anatomical landmarks in the brain MR images, along with their nearby image patches. Then, our LDMIL framework learns an end‐to‐end MR image classifier for capturing both the local structural information conveyed by image patches located by landmarks and the global structural information derived from all detected landmarks. We have evaluated our proposed framework on 1526 subjects from three public datasets (i.e., ADNI‐1, ADNI‐2, and MIRIAD), and the experimental results show that our framework can achieve superior performance over state‐of‐the‐art approaches.