Transfer Learning and Fusion Model for Classification of Epileptic PET Images

Epilepsy is a common neurological disease in China and it can be detected and diagnosed by PET images. For automatic classification tasks, it is essential to obtain discriminable features from medical images. The features of the pretraining neural network have been widely used in some image fields. In this paper, we propose a novel fusion modal transfer learning framework by three kinds of two-dimensional convolution networks (ResNet, VGGNet, Inception-V3) pretrained on ImageNet databases and a 3D convolution network of SVGG-C3D pretrained on the lung nodules databases of the Kaggle competition. Weights in the convolutional layers of each network were frozen to serve as fixed feature extractors. The multi-modal transfer learning framework on epileptic PET images is trained to extract features with the frozen weights. Combining four model characteristics, the weights of classifier (top layers) are trained to predict the epileptic and the normal. The proposed algorithm can be used to detect epilepsy more effectively than the deep learning model.