Feature Learning Using Stacked Autoencoder for Shared and Multimodal Fusion of Medical Images

In recent years, deep learning has become a powerful tool for medical image analysis mainly because of their ability to automatically extract more abstract features from large training data. The current methods used for multiple modalities are mostly conventional machine learning, in which people use the handcrafted feature, which is very difficult to construct for large training sizes. Deep learning which is an advancement in the machine learning automatically extracts relevant features from the data. In this paper, we have used deep learning model for the multimodal data. The basic building blocks of the network are stacked autoencoder for the multiple modalities. The performance of deep learning-based models with and without multimodal fusion and shared learning are compared. The results indicates that the use of multimodal fusion and shared learning help to improve deep learning-based medical image analysis.