X-ray image classification using domain transferred convolutional neural networks and local sparse spatial pyramid

The classification of medical images is a critical step for imaging-based clinical decision support systems. Existing classification methods for X-ray images, however, generally represent the image using only local texture or generic image features (e.g. color or shape) derived from predefined feature spaces. This limits the ability to quantify the image characteristics using general data-derived features learned from image datasets. In this study we present a new algorithm to improve the performance of X-ray image classification, where we propose a late-fusion of domain transferred convolutional neural networks (DT-CNNs) with sparse spatial pyramid (SSP) features derived from a local image dictionary. Our method is robust as it exploits the rich generic information provided by the DT-CNNs and uses the specific local features and characteristics inherent in the X-ray images. Our method was evaluated on a public dataset of X-ray images and was compared to several state-of-the-art approaches. Experimental results show that our method was the most accurate for classification.